setup a new scrapy project from cli

scrapy startproject first_scrapy

https://www.tutorialspoint.com/scrapy/scrapy_create_project.htm

sample program

import scrapy  

class firstSpider(scrapy.Spider): 
   name = "first" 
   allowed_domains = ["dmoz.org"] 
   
   start_urls = [ 
      "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/", 
      "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/" 
   ]  
   def parse(self, response): 
      filename = response.url.split("/")[-2] + '.html' 
      with open(filename, 'wb') as f: 
         f.write(response.body)


to run the program goto project directory and type 
"scrapy crawl 'name'"


name will be whatever is there in the program eg. here it is 'first'
as name = 'first'