This is a sample Scrapy project for educational purposes
Switch branches/tags
Nothing to show
Clone or download
Failed to load latest commit information.
quotesbot import quotesbot project Sep 27, 2016
.gitignore Initial commit Sep 27, 2016
LICENSE Initial commit Sep 27, 2016 point to spidyquotes repo in readme Oct 18, 2016
scrapy.cfg import quotesbot project Sep 27, 2016


This is a Scrapy project to scrape quotes from famous people from (github repo).

This project is only meant for educational purposes.

Extracted data

This project extracts quotes, combined with the respective author names and tags. The extracted data looks like this sample:

    'author': 'Douglas Adams',
    'text': '“I may not have gone where I intended to go, but I think I ...”',
    'tags': ['life', 'navigation']


This project contains two spiders and you can list them using the list command:

$ scrapy list

Both spiders extract the same data from the same website, but toscrape-css employs CSS selectors, while toscrape-xpath employs XPath expressions.

You can learn more about the spiders by going through the Scrapy Tutorial.

Running the spiders

You can run a spider using the scrapy crawl command, such as:

$ scrapy crawl toscrape-css

If you want to save the scraped data to a file, you can pass the -o option:

$ scrapy crawl toscrape-css -o quotes.json