A lightweight, fast, and automated web scraper built to extract product information (titles and prices) from a paginated e-commerce website.
🚀 Business Value
This tool was built to solve a common e-commerce problem: wasting time on manual data entry. Instead of spending hours manually copying competitor prices or supplier catalogs, this script does it in seconds and delivers a clean, ready-to-use dataset.
-
Automated Pagination: Navigates through multiple pages of the store automatically (e.g., handles 40+ pages in one run).
-
Data Cleaning: Strips hidden HTML characters, extra spaces, and newlines to ensure high data quality.
-
Direct CSV Export: Saves the final dataset directly into a structured .csv file, perfectly formatted for Excel, Shopify, WooCommerce, or any ERP system.
-
Anti-Blocking Measures: Uses custom User-Agent headers to simulate real browser traffic and prevent basic bot-blocking.
Language: Python 3.14
Core Libraries:
requests (HTTP requests handling)
BeautifulSoup4 (HTML parsing and data extraction)
csv (Data formatting and export)
The script accesses the target e-commerce category URL.
It iterates through all available pages automatically.
It extracts the raw HTML, locates the product containers, and isolates titles and prices.
It cleans the text and streams it line-by-line into ksiazki_helion.csv.
To run this script locally:
- Clone the repository.
git clone https://github.com/Twój-Nick/web-automation-tools.git- Navigate to the project folder:
cd web-automation-tools- Install required dependencies:
pip install requests beautifulsoup4Run the script:
python moj_scraper.py- Check the project directory – the generated ksiazki_helion.csv dataset will be ready for use.
This repository is for educational and portfolio purposes. Web scraping should be done ethically and in accordance with the target website's robots.txt policies.