Reconcile the formulation entity against what the solve actually ran

[kateebonner]

Important

Problem

The formulation entity is written once by the agent at the formulate stage and never checked against what actually ran: the authored solve script has no feedback path into the entity, so the formulation tab can silently disagree with the run forever. Observed in the wild — a workspace whose solve was a real CZ carries a formulation claiming target X, and nothing downstream will ever notice.

Approach

Make the run emit a small ground-truth manifest (target, trajectory type, key solve params) as a structured marker line in the run log — the same channel that already carries iteration/pulse markers — and have the extension reconcile it against the recorded formulation when it ingests run results: matching fields confirm, mismatching fields update the entity and append a provenance event flagging the correction.

Approaches Considered

• Solver-emitted manifest + ingest-time reconcile (chosen) — deterministic ground truth from the code that actually ran; reuses the existing structured-log-marker pattern.
• Heuristic parsing of the authored Julia script — rejected: fragile against free-form authored code, and the script isn't what ran if it was edited after launch.
• Record-time heuristics only (companion issue) — insufficient: record time can't see the script; the two guardrails cover different failure windows.

Scope

• In: manifest emission in the run harness; reconciliation + entity update + provenance event at result ingest; surfacing a mismatch note in the run entity/tool reply.
• Out: blocking or failing a solve on mismatch — the solve already ran; the data should follow reality.
• Out: reconciling the system entity (Hamiltonian/levels) — bigger surface, needs its own design.
• Out: UI changes — the formulation tab reads the corrected entity for free.

Assumptions / Open Qs

• Assumes the run harness (launcher + log conventions) is the right emission point and a marker line is acceptable to the existing log parsers.
• Open: on mismatch, silently correct + event, or keep both values and render a visible "formulation corrected from run" note in the tab?
• Open: what the manifest can honestly know for vetted vs. composed vs. free-tier scripts — free-tier authored code may need the emission helper injected.

---

Acceptance Criteria

• A completed solve whose manifest disagrees with the recorded formulation target results in an updated formulation entity plus an event recording old value, new value, and source (run reconcile).
• A matching manifest produces no entity change and no spurious event.
• Runs predating the manifest (no marker present) reconcile as a no-op — no warning, no change.
• The mismatch/correction is visible to the user in at least one surface (tool reply or run entity), not only in the event log.
• Tests cover mismatch, match, and missing-manifest paths.

Key Decisions

• Ground truth flows one way: run → entity. The entity is corrected to match reality, never the reverse.
• Reconciliation lives at result-ingest time in the extension plugin, not in the UI projection.
• Manifest is additive to the existing marker-line protocol; no new file format.

Source

• Found while diagnosing a wrong "x-gate" chip in the formulation tab: workspace cz-gate-transmon-2 (2026-07-24) recorded target X at formulate time and nothing downstream could ever contradict it.
• Companion issue: Warn on gate-target mismatch when recording a formulation #214 (record-time target-sanity warning) — covers the capture window; this issue covers everything after launch.