Ukraine-Russia War Web Crawler

A robust web crawler designed to gather information about the Ukraine-Russia war from various sources. This crawler is built with anti-detection mechanisms and human-like behavior simulation to bypass common anti-scraping measures.

Features

• Browser automation with undetected-chromedriver
• Human-like behavior simulation
• Anti-bot detection bypass
• Dynamic content handling
• Article extraction and processing
• Local storage in JSON/CSV format

Project Structure

src/
├── crawler/         # Core crawler implementation
├── utils/          # Utility functions
└── config/         # Configuration files
tests/              # Test files

Setup

1. Create a virtual environment:

python3 -m venv venv
source venv/bin/activate  # On Unix/macOS

2. Install dependencies:

pip install -r requirements.txt

Usage

[Usage instructions will be added as the project develops]

License

MIT License