Interpol Notice Scrapers (Red & Yellow)

Python scrapers for collecting all public notices from Interpol's API. Includes two independent scripts:

1. main.py - Red Notices (wanted persons) with parallel detail fetching
2. yellow_notice.py - Yellow Notices (missing persons) with auto-verification

Both scrapers use a recursive country/sex/age splitting strategy to bypass the API's 160-result pagination limit and achieve exhaustive collection.

Setup

pip install beautifulsoup4 pandas

Usage

Red Notices

python main.py [OPTIONS]

Argument	Default	Description
--output	interpol_red_notices.csv	Output CSV path
--workers	20	Parallel threads for detail fetching
--delay	0.5	Delay between API calls (seconds)
--max-pages	None	Limit Phase 0 pages (for testing)

# Default run
python main.py

# Custom: 30 workers, specific output
python main.py --workers 30 --output red_notices.csv

Yellow Notices

python yellow_notice.py

Runs all phases automatically. Output files:

• interpol_yellow_smart_all.csv - Main data
• yellow_missing_report.csv - Completeness report
• interpol_yellow_smart_all_final.csv - Final consolidated data after catch-up

Architecture

Shared (scraper_utils.py)

Common utilities used by both scrapers:

• HTTP client with retry-safe JSON fetching
• Notice deduplication by entity ID / URL / composite key
• Paginated fetching with automatic country > sex > age splitting
• CSV writer

Red Notices pipeline

1. Phase 0 - Global page scan to collect notice URLs
2. Phase 1 - Per-country scan to find notices missed by global pagination
3. Phase 2 - Parallel ThreadPoolExecutor to fetch and normalize detail pages (age calculation, country name conversion, infraction classification)

Yellow Notices pipeline

1. Scrape - All 676 two-letter country codes, by nationality and birth country
2. Verify - Compare local counts against API totals
3. Catch-up - Re-fetch incomplete countries without filters