Release evidence files mandated by three docs were never committed

[dichovsky]

Three docs require committed release evidence: docs/design/product-spec.md:117-119 ("Each release … commits docs/release/artifacts-<date>.json"), docs/release/README.md ("This directory holds the evidence files"), and docs/release/live-smoke-checklist.md:73 (a checklist item). But docs/release/ contains only three .md files and git log --all -- 'docs/release/artifacts-*' is empty — even though v0.1.1 is tagged and published.

Impact: the release-gate evidence trail the docs mandate does not exist for either shipped release, so the documented release process and actual practice have already diverged.

Acceptance: either the evidence file is committed for the next release (with the 0.1.0/0.1.1 gap back-filled or annotated), or the product-spec/checklist requirements are amended to match actual practice.

[dichovsky]

Assessment: re-verified against main @ 97d7f88. Confirmed as-is, and the scope is wider than filed — five documents mandate or describe the evidence file, not three.

Confirmed as filed

• docs/design/product-spec.md:118-119 — "Each release re-runs the credentialed live smoke and commits docs/release/artifacts-<date>.json", inside what :107 calls the canonical release-gate list that "[o]ther documents reference … and must not keep a separate copy".
• docs/release/README.md:3-4 — "This directory holds the evidence files the maintainer produces when releasing crew (release Gates 2 & 3 …)", with artifacts-<date>.json specified at :16-23 and "commit the dated evidence file" at :32-34.
• docs/release/live-smoke-checklist.md:73 — - [ ] artifacts-<date>.json is committed under docs/release/.
• docs/release/ holds exactly three files: README.md, live-smoke-checklist.md, publishing.md. git log --all -- 'docs/release/artifacts-*' is empty across all refs.
• package.json version is 0.1.1; CHANGELOG.md:29 and :38 record [0.1.1] — 2026-07-13 and [0.1.0] — 2026-07-13. Two releases shipped, zero evidence files.

Three deltas against the body (noted here; body left as written)

1. Two further mandates the body does not name. docs/release/publishing.md:48-49 is step 4 of "Cutting a release (every version)": "Commit the evidence: docs/release/artifacts-<date>.json, and update the gate statuses". And docs/design/testing-strategy.md:227-230 requires the release smoke to "[s]tore the tool version, date, artifact hash, and result as a release artifact (a generated file kept as evidence)". Any fix has to move all five documents together or it just relocates the contradiction.
2. The file was never produced-and-ignored — it was never committed. git check-ignore -v docs/release/artifacts-2026-08-06.json reports not ignored, and no docs/release or artifacts-*.json pattern exists in .gitignore. So the gap is a skipped process step, not a tooling accident.
3. The line citation is :118-119, not :117-119 — :117-118 is the doctor version-floor clause; the evidence sentence begins mid-:118.

Also worth recording, because it bounds the damage: no closed-gate claim is falsified here. product-spec.md:112-113 closes only the SQLite-stress and Security-review gates; the Participant-CLI-matrix and Model-Backend gates (Gates 2 & 3 — the ones this evidence closes) are described at :116-119 as having only their machine-runnable half landed. This is a process that was documented and then not followed, not a gate recorded as closed against missing evidence. (That second, sharper failure does exist — it is #37's SQLite-stress finding — and it is not this issue.)

Verdict: PLAN-ONLY

• Blast radius — five documents: docs/design/product-spec.md (the canonical release-gate list), docs/design/testing-strategy.md, docs/release/README.md, docs/release/live-smoke-checklist.md, docs/release/publishing.md. Nothing under src/, bin/, web/, or docs-site/, so no module boundary in CLAUDE.md is crossed and coverage cannot move. tests/tools/release-smoke.test.ts:108-112 is the producer (join(repoRoot, 'docs', 'release'), artifacts-${isoDate()}.json) and would need a change only under Branch B. Nothing currently guards any of this: no test reads docs/release/, and collectAdrs/collectCommands in tests/unit/docs-facts.test.ts read ADR H1s, package.json, and vitest.config.ts — never the release docs. The drift is unguarded in both directions.

• Reversibility — the prose is a text revert, but the artifact is not. A committed artifacts-<date>.json becomes part of the permanent release-evidence trail for a published version; and amending the requirement away after 0.1.0/0.1.1 have already shipped retroactively changes the standard those two releases were held to. Neither half is cheaply undone once merged, and neither is undone at all once it is the record a later release cites.

• Contract impact — no ADR, no FR-*/NFR-*, no CLI grammar, no record type, no error code, no data-model or State Store change; CURRENT_SCHEMA_VERSION stays 7. product-spec.md and the docs/release/ files sit outside the four-tier authority order in docs/README.md:9-12. But the release-gate table is explicitly canonical, and docs/README.md's "Current readiness" is the single source of truth for release status — so an edit here changes what "Gate 2/3 closed" will mean for every future release, which is a governance change wearing a documentation change's clothes.

Why not IMPLEMENT. Both branches of the Acceptance are blocked on facts or authority this routine does not have:

• Branch A cannot be executed here. The recorder needs the real Participant CLIs installed (publishing.md:38-39) and the interactive half needs a human at a keyboard with real credentials (live-smoke-checklist.md:3-5). Generating an artifacts-*.json from this container would fabricate the exact record that closes Gates 2 & 3. Not a judgment call — it must not happen.
• Back-filling or annotating the 0.1.0/0.1.1 gap needs a fact only the maintainer holds: whether the live smoke was actually performed for those releases and merely not committed, or never performed at all. Either sentence is a claim about release history; writing the wrong one puts a false statement into the release record permanently.
• Branch B is a policy reversal, not a correction. Deleting "commit the evidence" from five documents decides that crew's release process no longer keeps an evidence trail. That is a wontfix-adjacent judgment about a release gate, which .github/ISSUE_TRIAGE.md routes away from the routine.

I could execute either branch cleanly. Choosing between them is not mine to choose.

---

The decision a human must make first

Is committed per-release evidence still a requirement of crew's release process — and was the live smoke actually run for 0.1.0 and 0.1.1?

The second half is not optional colour: it decides whether the gap gets annotated as "run, not committed" or "not run", and those are different statements about two published releases. Everything below follows mechanically from the two answers.

---

Branch A — keep the requirement, close the gap honestly

Chosen if the answer is "yes, evidence is required".

1. docs/release/README.md — add an Evidence on file section after Contents, as a ledger with one row per released version: version, date, whether artifacts-<date>.json is on file, and for the gaps a one-line reason in the maintainer's words. Seed it with 0.1.0 and 0.1.1 using the answer to the question above. This is the file that claims to hold the evidence, so it is the right place to state what is and is not held.

2. docs/design/product-spec.md:118-119 — the requirement sentence is not edited; it is correct under this branch. Add one clause pointing at the README ledger so the canonical gate list names where evidence status is tracked without keeping a second copy of it (:107-108 forbids a separate copy).

3. docs/release/live-smoke-checklist.md:73 and docs/release/publishing.md:48-49 — unchanged. Both are already correct under this branch.

4. docs/design/testing-strategy.md:227-230 — unchanged.

5. The evidence file itself is out of scope for any routine PR and belongs to the next release cut, per publishing.md:35-49. Say so explicitly in the PR body rather than leaving a reader to wonder why the fix commits no artifact.

Tests. A guard belongs here, but it must not be the obvious one. Asserting "every dated CHANGELOG.md release heading has a matching docs/release/artifacts-*.json" fails on current main for both 0.1.0 and 0.1.1 — correctly, which is the problem: it turns main permanently red for a gap that can only be closed by a human at a keyboard, and CI red-on-main is not a state to merge into. Two workable shapes:

• tests/unit/release-evidence.test.ts asserting that the README ledger names every released version in CHANGELOG.md — not that an artifact exists. It fails before the fix because the ledger does not exist at all; it stays green through the gap and still catches the next release that forgets to record its status.
• Defer the artifact-existence assertion to the release cut that first commits one, and note in the ledger that it is deferred.

Whichever is chosen, state in the PR body which one and why — a reviewer will otherwise read the weaker assertion as an oversight.

---

Branch B — narrow the documented process to actual practice

Chosen only if the answer is "no, committed evidence is not required". Larger than it looks: five files, and it changes what Gates 2 & 3 mean.

1. docs/design/product-spec.md:118-119 — "Each release re-runs the credentialed live smoke and commits docs/release/artifacts-<date>.json" → the live smoke is re-run per release and its record is produced locally, naming explicitly what is retained and where. Do not touch :112-113's closed-gate sentence: it names the SQLite-stress and Security-review gates only, and neither is affected. Do not touch the Gate 2/3 rows of the table at :124-131 — the evidence required for those gates is the smoke test itself, not its committed transcript, and conflating the two would narrow the gates rather than the paperwork.

2. docs/release/README.md — :3-6 ("holds the evidence files") is false under this branch and needs the largest rewrite of the five: the directory holds the procedure, and the recorder writes a local, uncommitted capture. :16-23's artifacts-<date>.json description stays (it still describes real output) but must be re-framed as local. :32-34's "commit the dated evidence file" goes.

3. docs/release/live-smoke-checklist.md — :73 is deleted or reworded; :22 ("record the outcome … alongside the generated artifacts-<date>.json") must say where that record then lives, or the checklist loses its output entirely.

4. docs/release/publishing.md:48-49 — step 4 loses "Commit the evidence"; the gate-status update in the same step stays.

5. docs/design/testing-strategy.md:227-230 — "kept as evidence" is the phrase that goes stale; restate as a locally generated record.

6. .gitignore — add docs/release/artifacts-*.json, so the recorder's output cannot be committed by accident after the docs stop asking for it. Today it is explicitly not ignored (verified above).

7. tests/tools/release-smoke.test.ts:108-112 — the output path is unchanged; it still writes into docs/release/. Consider whether it should instead write outside the tree now that the file is never committed. That is a real decision, not a cleanup — flag it, do not fold it in silently.

Tests. Same tests/unit/release-evidence.test.ts, inverted: assert that docs/release/artifacts-*.json is git-ignored (git check-ignore, or the pattern's presence in .gitignore). It fails before the fix because the path is not ignored today.

---

Out of scope, either branch

• The SQLite-stress gate's macOS evidence — ADR-0009 stress coverage claims exceed what stress.yml actually runs #37 (triage/plan-ready). Same document, different gate, and that one does falsify a closed-gate claim.
• docs/README.md's "Current readiness" release status — Release status stale: 0.1.1 shipped but docs still record 0.1.0 #38 (PR docs: record 0.1.1 as the published release status #94 open). It records which version shipped, not what evidence backs it.
• The release-gate table rows themselves (product-spec.md:124-131). Neither branch changes what any gate requires; both change only how its record is kept. An edit that lands in that table means the change has drifted past this issue.
• Actually running the live smoke, or generating any artifacts-*.json. Requires eight installed Participant CLIs, two Model Backends, and a human. No routine PR produces this file.

Label applied: triage/plan-ready. No code written.

---

Generated by Claude Code