Amazon-scraper is a command line application to collect reviews and questions/answers from amazon products.
Read the documentation here: amazon-scraper on readthedocs
Table of contents
cloning this repository
$ git clone https://github.com/picorana/amazon-scraper.git $ cd amazon-scraper $ pip install -r requirements.txt $ python setup.py install
amazon-scraper via command line by running
$ amazon-scraper [asin]
asin is a unique identifier for a product on amazon. You can find it in the url:
A query to
https://www.amazon.com/gp/product/B01H2E0J5M would look like this:
$ amazon-scraper B01H2E0J5M
You can also insert multiple asins:
$ amazon-scraper B01H2E0J5M B01GYLZD8C B0736R3W1F
or load them from file:
$ amazon-scraper --file asins.txt
the file needs to have each asin on one line, like this:
B01H2E0J5M B01GYLZD8C B0736R3W1F
amazon-scraper downloads pages, reviews, questions and answers.
It will save its output in folders:
pages will contain the main pages of the product, useful for extracting more info about the product.
You can disable this function by using the option
results will contain the reviews, organized in json files.
You can disable scraping the reviews by using the option
questions will contain the questions and answers, organized in json files.
You can disable scraping the questions by using the option
positional arguments: asin Amazon asin(s) to be scraped optional arguments: -h, --help show this help message and exit --file FILE, -f FILE Specify path to list of asins --save-main-pages, -p Saves the main pages scraped --verbose, -v Logging verbosity level --no-reviews Do not scrape reviews --no-questions Do not scrape questions