README.md

Webscraper v3

Webscraping application made in Python to process online marketplace data for different products on various platforms
Uses Selenium and Chrome driver to open webpages in a headless (invisible) browser and access their contents
Currently scrapable: Amazon India, Flipkart, BigBasket

USAGE

Setup

Required on your system:

• Python (added to PATH): Install from https://www.python.org/downloads/ and add to PATH variable
• Chrome: Install from https://www.google.com/intl/en_us/chrome/
• Chrome Driver: Download (same version as Chrome!) from https://chromedriver.chromium.org/downloads (versions <= 114) or https://googlechromelabs.github.io/chrome-for-testing (versions >= 115)
• Selenium module for Python: Run command pip install selenium

Edit chrome_driver_path in consts.txt according to your system

Running

Windows users can quickly run by clicking on RUN.bat
Otherwise, run commands cd path/to/WebScraper and python main.py
If cloned with Git, Windows users can use UPDATE.bat to pull the latest version while preserving consts.txt

Notes

Selenium handshake failure errors can mostly be ignored