NSA Codebreaker Challenge Scraper

Scripts to scrape and analyze NSA Codebreaker Challenge leaderboards and personal submissions.

Installation

cd cbc_scraper

# Using uv (recommended)
uv venv && source .venv/bin/activate && uv pip install -e .

# Using pip
python -m .venv venv && source .venv/bin/activate && pip install -e .

Usage

Leaderboard Scraper

python scrape_leaderboards.py                    # Scrape current year
python scrape_leaderboards.py --all-years        # Scrape 2018-2025
python scrape_leaderboards.py --year 2023        # Scrape specific year
python scrape_leaderboards.py --display          # Display cached data

Output:

• data/leaderboard_stats_2025.json - Current year
• data/archived_leaderboards.json - Historical years (2018-2024)

Features:

• Color-coded solve rates (🟢 ≥25% | 🟡 2-25% | 🔴 <2%)
• Fast parallel scraping (~2-3s current year, ~15-20s all years)
• No authentication required

Submissions Scraper

python scrape_submissions.py            # Scrape your submissions
python scrape_submissions.py --display  # Display cached data

Authentication (prompts for input if not set):

export CBC_EMAIL="your.email@example.com"
export CBC_PASSWORD="your_password"

Output:

• data/submission_stats.json - Your submission history with time spent per task

Features:

• Calculates time spent on each task
• Shows attempts per task
• Tracks progress on current task

Time Calculation:

• Pass/Fail Detection:
  • Tasks 0-6: Compares response message hash against known success hashes
  • Task 7: Checks if latest submission lacks the failure message
• Time Tracking:
  • Latest uncompleted task: now - previous_task_completion (tracks ongoing work, even with 0 submissions)
  • Passed tasks: task_completion - previous_task_completion
  • First task: last_submission - first_submission