ObserverBench v0.1.0

ObserverBench v0.1.0

ObserverBench tests whether an internal estimate is adequate for the action it
will guide. This first public release includes:

• closed-loop control tasks with a checked external-observer adapter;
• finite-effect prediction tasks for GPT-2-small IOI and a frozen
  Qwen2.5-7B induction-copy study;
• safety-triage tasks with fixed intervention or audit budgets;
• TaskCard, ObserverCard, prediction-table, and task-pack contracts;
• checked reference results and task-specific leaderboard tables;
• paper source, reproduction scripts, and CPU-safe tests;
• a GitHub-native, data-only submission preflight that never executes
  contributor code.

The public launch automates submission preflight. Sealed scoring and automatic
leaderboard ranking require a separately configured private evaluator and are
not enabled in v0.1.0.

Verification on a clean archive: 301 tests passed and 11 were skipped because
they require expensive model runs or separately archived raw experiment
bundles. The paper and static site also build successfully.

Archived release DOI: https://doi.org/10.5281/zenodo.22136091

Mechanistic Tomography, the measurement framework that motivates
ObserverBench: https://arxiv.org/abs/2608.19338