Skip to content

FlightGear ATC clearance ledger — retained calibration trajectories

Latest

Choose a tag to compare

Retained calibration trajectories for the FlightGear ATC clearance-compliance
ledger task (PR PhiloLabs#105).

Eight raw harness trajectories, kept as bytes rather than numbers alone:

  • five anti-shortcut ablation rollouts (ablation-*);
  • two replaced rounds (replaced-round_*) — see below;
  • one required-row full trajectory (required-row_claude-opus-4.8-native_full-trajectory.jsonl.gz).

Each agent's final submitted ledger is also in the PR itself, under
calibration/ablations/ledgers/ and calibration/rollouts/ledgers/, so the
scored table can be re-derived with calibration/rescore_ledgers.py without
downloading anything here.

Replaced round 1antigravity-gemini-3.6 at 2026-08-12T20:35:42Z stopped
on a provider resource-exhausted error partway through media extraction and
produced no ledger; it was re-run to completion on 2026-08-13, and that
completed run is the one reported as the required Antigravity row.

Replaced round 2claude-opus-4.8-agent-host-parity is the Opus 4.8 round
that ran through a VS Code Copilot agent-host whose wrapper prompt added solving
guidance. It is replaced for harness parity by the native claude -p row, not
because it failed: it scores 0.6060 on the shipped judge, well above the
required rows, and is retained precisely so that number stays checkable.

Required-row full trajectory — the native claude -p rollout is 15,343,222
bytes raw (sha256 819b958515268e0ccfc68c48d87d309d129b5b549eb9991af75b84d0b08a8990).
The copy committed in the PR is condensed to 1,232,159 bytes by dropping the
redundant stream_event SSE deltas and replacing base64 screenshot payloads
with content-addressed placeholders. Regenerate the committed file byte for byte
from this asset:

gzip -d required-row_claude-opus-4.8-native_full-trajectory.jsonl.gz
python3 calibration/rollouts/condense_trajectory.py \
    required-row_claude-opus-4.8-native_full-trajectory.jsonl out.jsonl
sha256sum out.jsonl   # be4a275123397f16dea1632940aae991e0136edfec41949ba29a3fff9cdbc8df

SHA-256 of every asset is listed in calibration/ablations/README.md (ablations
and replaced rounds) and calibration/rollouts/README.md (required row).