Building AI agents for the ARC-AGI-3 interactive reasoning benchmark.
ARC-AGI-3 evaluates AI agents on four core capabilities in novel, dynamic environments:
- Exploration - Actively obtain information by interacting with the environment
- Modeling - Turn observations into generalizable world models
- Goal-setting - Identify desirable states without explicit instructions
- Planning & Execution - Map action paths and course-correct on feedback
Environment: 64x64 grid, 16 colors, 7 standardized actions (RESET, ACTION1-6, ACTION7/Undo)
Scoring: RHAE (Relative Human Action Efficiency) — level_score = (human_actions / ai_actions)^2
# Install dependencies
uv sync
# Verify SDK works (offline mode)
uv run python exploration/game_explorer.py --list
# Run random baseline agent on a game
uv run python -m agents.random_agent --game ls20
# Run evaluation across all local games
uv run python evaluation/benchmark.py --agent randomagents/ — Agent implementations (base class, random, heuristic, etc.)
core/ — Shared utilities (state handling, memory, pattern recognition, search)
exploration/ — Environment exploration tools and Jupyter notebooks
evaluation/ — Benchmarking and RHAE scoring
kaggle/ — Kaggle submission templates and packaging
data/ — Replays, trained models, and local data
- No internet during Kaggle evaluation — cannot call external APIs
- 6-hour runtime limit (CPU or GPU notebook)
- Open source required for prize eligibility
- Efficiency matters — RHAE uses squared ratio, so 2x more actions = 0.25 score
| Milestone | Date |
|---|---|
| Competition Start | 2026-03-25 |
| Milestone 1 | 2026-06-30 |
| Milestone 2 | 2026-09-30 |
| Team Merge Deadline | 2026-10-26 |
| Final Submission | 2026-11-02 |
| Results | 2026-12-04 |