Skip to content

ScreenQuest Arena v0.1.0 — measured game-agent leaderboard

Choose a tag to compare

@DanielTea DanielTea released this 04 Oct 11:30

ScreenQuest Arena v0.1.0 is an open-source reference implementation for pixels-only game-agent evaluation.

Other benchmarks test productivity/usefulness, we test intelligence.

  • Ten runnable scenarios: two original 2D games and eight ViZDoom scenarios.
  • 400 measured baseline episodes, plus 36 exhibition episodes comparing actual GPT-6 Astra, Claude Opus 5 and local reference policies.
  • A strict per-decision <100 ms gate; slower model runs stay in exhibition.
  • Complete screenshot/action evidence, deterministic replay, fixed-suite scores and bootstrap intervals.
  • 49 passing local tests and automated Linux checks.
  • A public leaderboard, participant protocol, submission form and 42 researched game candidates/families.

All results are local-unattested. The model exhibition uses two seeds and an eight-decision horizon; it is an integration smoke test, not a robust model ranking. The simulation is lockstep, not continuous realtime. Modern AAA adapters and independently isolated official evaluation remain roadmap work.

The 204 MiB evidence archive includes all final-campaign observations and traces. Verify it using SHA256SUMS and the instructions in results/PROVENANCE.md. Third-party rendered game content retains its own rights; the MIT license covers original code.