Found while completing the Verdict → Freshness rename (PR #2, 2026-07-25).
What happened
src/session/__fixtures__/run-bundle.v1.schema.json is a hand-copied snapshot of the canonical schema published at chatwright/chatwright/formats/run-bundle/v1/schema.json. Nothing checks that the copy still matches.
So when the canonical schema changed (verdict → freshness, still a required property), this repo kept validating its bundle output against the old snapshot — which of course still matched the code, because the code had not changed either. The schema-validation test went on passing while the runtime emitted a field the published schema no longer allows.
Proof that the test itself was fine: refreshing the vendored fixture without touching the writer made it fail immediately with required/additionalProperties errors. The guard was correct; the thing it guards against had been copied into the guard.
Why it matters
The bundle format is the interop surface between the Go runtime, this runtime and the Studio player. A stale local copy means this repo can drift away from the published format and its CI will report green — which is the specific failure mode a schema check exists to prevent.
There are currently three copies of this schema in the fleet:
| Location |
Drift-guarded? |
chatwright/chatwright/formats/run-bundle/v1/schema.json (canonical) |
✅ CI gate against sdk-go |
chatwright/studio/worker/formats/run-bundle/v1/schema.json |
⚠️ manual |
chatwright/runtime-ts/src/session/__fixtures__/run-bundle.v1.schema.json |
❌ none |
Suggested fix
A CI step that fetches the canonical schema and fails on any difference from the vendored copy — the same shape as the existing format-drift workflow in chatwright/chatwright. Cheap, and it converts a silent divergence into a loud one.
Worth considering the Studio copy in the same pass.
Found while completing the
Verdict→Freshnessrename (PR #2, 2026-07-25).What happened
src/session/__fixtures__/run-bundle.v1.schema.jsonis a hand-copied snapshot of the canonical schema published atchatwright/chatwright/formats/run-bundle/v1/schema.json. Nothing checks that the copy still matches.So when the canonical schema changed (
verdict→freshness, still a required property), this repo kept validating its bundle output against the old snapshot — which of course still matched the code, because the code had not changed either. The schema-validation test went on passing while the runtime emitted a field the published schema no longer allows.Proof that the test itself was fine: refreshing the vendored fixture without touching the writer made it fail immediately with
required/additionalPropertieserrors. The guard was correct; the thing it guards against had been copied into the guard.Why it matters
The bundle format is the interop surface between the Go runtime, this runtime and the Studio player. A stale local copy means this repo can drift away from the published format and its CI will report green — which is the specific failure mode a schema check exists to prevent.
There are currently three copies of this schema in the fleet:
chatwright/chatwright/formats/run-bundle/v1/schema.json(canonical)chatwright/studio/worker/formats/run-bundle/v1/schema.jsonchatwright/runtime-ts/src/session/__fixtures__/run-bundle.v1.schema.jsonSuggested fix
A CI step that fetches the canonical schema and fails on any difference from the vendored copy — the same shape as the existing
format-driftworkflow inchatwright/chatwright. Cheap, and it converts a silent divergence into a loud one.Worth considering the Studio copy in the same pass.