Web Scraper Suite

Overview

This repository contains a suite of web scrapers developed using puppeteer for both static and dynamic web pages. The scrapers are designed to extract data from various websites, providing a flexible solution for web data extraction tasks.

Features

• Scrapers for both static and dynamic websites
• Utilizes Selenium for dynamic page interaction
• Extracts data from multiple websites
• Easy-to-use interface
• Customizable scraping parameters

Installation

1. Clone the repository to your local machine.
2. Install the required dependencies using npm:
3. Download the appropriate web driver for your browser and ensure it's in your system PATH.

Usage

1. Configure the scraping parameters in the scraper scripts according to your requirements.
2. Run the desired scraper script using node:
3. The scraped data will be saved in the specified output format (e.g., CSV, JSON).

Websites Scraped

• www.politifact.com
• www.altnews.in
• mastodon.social

Blog

For more insights into web scraping techniques, tips, and tutorials, visit medium.com/@aymenmehmood812.

Contributors

• Aymen Mehmood

License

This project is licensed under the MIT License - see the LICENSE file for details.