Demonstration

• We produced the code and tested a scrapy shell.
• We ran the code or spider.
• We divided it into two forms for developing the project, without venv or with venv.

Without venv : We should enter the path of the project or spider for executing the command.

scrapy crawl w3wiki_article

With venv: We could execute the command in any path if we are in mode venv.
 

scrapy crawl w3wiki_article

We can store the data in a file, with the commands below:

scrapy crawl w3wiki_article -O w3wiki_article.csv 

or

scrapy crawl w3wiki_article -o w3wiki_article.csv

O(create and insert in a new data file) and o(create and append in a new data file) are commands to create a new file and insert. 
 

Outputs:

Let’s suppose we are reading some content from a source like websites, and we want to save that data on our device. We can copy the data in a notebook or notepad for reuse in future jobs. This way, we used scraping(if we didn’t have a font or database, the form brute removes the data in documents, sites, and codes).

But now there exist many tools for scraping one site. However, this example was choosing Scrapy for scraping Python Site. Scrapy is a framework that extracting data structures or information from pages.