feat(guards): enforce the apparatus, retire the baselines, triage the debt (#327)

[m2ux]

Closes #327.

Corpus halves: #333 (merged) and #335 (open). This PR's submodule pointer pins #335's tip, so #335 merges first.

Rebased onto main@85e89edd after #332 landed. Two conflicts, both resolved: docs/development.md (my rewrite of the Validating Workflows section replaced the per-guard command list #332 had just amended — the fact it added, that check:technique-template also validates artifact bodies, is preserved on that guard's registry entry), and the workflows submodule pointer. Rebasing also surfaced six findings that #331's corpus work had independently fixed; the stale-entry check named all six on the first run, and they are pruned in a dedicated commit.

The repo grew 17 guard scripts in five weeks. CI ran four checks, 15 guards ran only when someone remembered, and four guards wrote what they found into stored baselines that nothing revisited — so the apparatus found bugs and then lost them. Both of #324's headline defects had been sitting in binding-fidelity-baseline.json, detected and silenced, while they shipped and burned a three-hour live run.

R1 — merge-base deltas instead of stored baselines

scripts/check-delta.ts resolves the merge-base with the integration branch, materialises it in a throwaway worktree with the workflows submodule pinned to the commit that tree recorded, runs the guard registry against both trees, and reports only the difference — including what your change fixed.

Nothing is stored, so nothing drifts, no N fixed noise, no baseline-update PRs, and the verdict is always exact and scoped to the change. Base results cache under .guard-cache/ keyed by (base commit, base corpus commit), so the doubled runtime is paid once per rebase rather than once per run. It also removes the stash / re-run / unstash / re-run dance that happened three separate times during the #324 session.

Running it on the change that introduces it surfaced three ways a cross-guard-version comparison can mislead, all now handled: a guard on the finding protocol in HEAD but not in base compares verdicts rather than reading every JSON line as a finding ([PROTOCOL]); a base that cannot run a guard as invoked means there is no before-state, not a hole in the delta ([NO-BASE]); and on the coarse path new output lines are shown but not counted, because without a finding protocol a format tweak is indistinguishable from a finding.

Proven end to end: a deliberately bogus step-binding input key in the corpus reports as exactly one introduced finding; reverting it returns the delta to clean.

R2 — CI enforcement

.github/workflows/verify.yml runs typecheck, test:ci, and check:all on every pull request.

Two constraints shaped it: "test": "vitest" is watch mode, so a test:ci script was added rather than relying on vitest's CI detection; and both submodules are branches of this repo behind an SSH URL the default token cannot use, while .engineering nests a private repo CI cannot clone. The corpus is therefore a second actions/checkout of the pinned gitlink (git rev-parse HEAD:workflows) — no submodule machinery, no URL rewriting, no credential handling, and .engineering stays out. It verifies the corpus commit the tree under review actually adopts, not the branch tip.

R3 — triage the suppressed findings

Every remaining finding carries a verdict and a named rationale in scripts/binding-fidelity-triage.json:

Verdict	Count	Guard behaviour
harmless	70	correct by design — suppressed
fix-later	125	real debt — suppressed, counted in the summary; tracked as #336
live-bug	0	reported; the guard stays red until fixed
untriaged	0	reported
stale	0	reported

Rationale classes: shared-op-return-contract, shared-op-caller-argument, pattern-library-seed, artifact-template-placeholder, documentation-notation, external-tool-syntax, verdict-component-of-composite (harmless); undeclared-seed, terminal-product-unconsumed (fix-later). A finding absent from the file is untriaged and reported; an entry matching nothing is stale and reported. There is no --update-baseline — a verdict is a human judgement, which is exactly what the retired baselines let a regenerate flag skip. --emit-untriaged prints what still needs one, --emit-all prunes what no longer occurs.

One class burned down

verification-result-unconsumed — an operation returns a verdict and no gate, condition, or validate target reads it — held 10 findings and is now empty. Eight are fixed in the corpus (#335): three verdicts gained a validate action, two gates now read the declared verdict instead of a parallel probe, and three outputs the checkpoint actually owns were removed from their technique. One was a guard blind spot (below) and one was misclassified by the predicate-name heuristic.

Two of the three enforced verdicts were security seams: remediate-vuln computed security_remote_configured and origin_untracked and discarded both, so a stealth run's isolation rested on a human reading git remote -v.

check:review-mode follows the same shape with three accepted instances in ACCEPTED_HEADLESS_AUTO_ADVANCE, each with the reason it is safe under the workflow's declared headless-review design.

229 → 195 was mostly guard reach, not paydown

Five blind spots were making the count dishonest:

• A step's when is a bare expression over bag names (has_debt_markers == true), so an output whose only consumer was a when gate read as dead. Same shape as the validate target case below; both now route through one expression scan. The reverse direction — resolving those names, so a gate naming an unproducible value is a finding — is Guard reach: a gate expression is counted as consumption but never resolved #341.

• The activity walk was non-recursive, so meta/activities/patterns/ had never been measured at all — five borrowable pattern activities, invisible in both directions. That is how concurrency: concurrency (a binding to an unproducible bag name) survived.

• Reads were scanned inside fenced code blocks, so an artifact template's {impact} — a field of the rendered report — read as an unresolved bag read. Inline code spans are deliberately still scanned: house style backticks real designators.

• A validate action's target expression didn't count as consumption, so an output whose only consumer is a validate gate read as dead. The opposite of dead.

• The shared OPTIONAL_INPUT_RE matched *(optional)* but not *(optional, default X)*, so inputs declaring both read as required-and-unsupplied — on the guard and on the server's provenance annotation, which shares the constant.

Three live defects found and fixed in #333: headless review runs auto-creating GitHub/Jira issues, the stealth push path discarding push_remote_verified and commits_signed, and substrate-node-security-audit requiring user_request without declaring it.

The stale-entry check has now earned its keep twice: it named the nine entries the verdict burn-down closed, and — after rebasing onto main — the six that #331's corpus work had independently fixed. A baseline would have kept all fifteen; binding-fidelity's carried 27 such entries, overstating the recorded debt by ~12%.

R4 — provision worktrees so they can measure

scripts/provision-worktree.sh checks out the submodules a worktree records and makes node_modules resolvable. Idempotent, and the resolvability test is require.resolve("tsx/cli") rather than the presence of a node_modules directory — vitest leaves a bare node_modules/.vite behind that would otherwise read as provisioned.

R5 — delete the baselines

All four are gone. audience and identifier-qualification are hard zero. review-mode-gating keeps three reasoned acceptances in code. binding-fidelity's 1282-line baseline is replaced by the triage file.

S1 — one registry, one sweep

scripts/guards.ts is the single enumeration of what verifies this repo, and check:all walks it: 17 guards, one table, ~1.2s. Adding an entry enforces the guard in check:all, in check:delta, and in CI.

  [PASS] binding-fidelity            0.9s  OK — no live or untriaged binding defects (210 triaged as accepted debt)
  [PASS] identifier-qualification    0.4s  OK — every technique I/O id is a qualified noun phrase
  ...
17 guard(s) in 1.2s — 17 pass, 0 fail, 0 unmeasured

The three scripts package.json never invoked are registered as check:refs, check:activities, check:workflow-yaml. validate-workflow-yaml gained a corpus-wide default instead of demanding a positional path, and check-all-refs now exits 1 on an unresolved reference — it used to print total unresolved: N and exit 0.

S2 — fail loud instead of passing vacuously

One shared protocol (scripts/guard-protocol.ts) with three exit codes: 0 clean, 1 findings, 2 could not measure. check:all never folds exit 2 into a pass.

requireWorkflowsRoot rejects a missing, non-directory, or workflow-less root and names the knob that selected it; assertScanned fails a guard that inspected nothing. check-prism-lens-reachability and analyze-io-protocol-refs stopped hardcoding the path, check-stealth-isolation dropped its private copy of the precedence rules, and the 14 pinned corpus paths across 13 test files route through tests/corpus-root.ts.

check-site-links and check-svg-layout are correctly not corpus-scoped — they read site/ — so the issue's S2 surface count of 5 is really 3 plus those two.

S3 — corpus-SHA-stamped baselines

tests/e2e/__snapshots__/corpus-sha.json records the corpus commit the walk snapshots were generated against, and a mismatch fails naming both SHAs with the remedy. Six e2e tests were red for exactly this reason during #324, and diagnosing it produced nothing. npm run baseline:stamp updates it alongside a submodule bump.

S4 — bench:token scope

--workflow (default work-package), recorded in the output, and a comparison across two different workflows is reported but can never pass --gate — the same discipline already applied to context mode. It previously walked one workflow structurally and reported a confident delta for a change it could not see.

S5 — cost, re-measured

	Before	Now
Full guard sweep	14 hand-chained invocations, 14 output formats	npm run check:all — 1.2s
"Did I cause this?"	stash → run → unstash → run → diff by hand	npm run check:delta, base half cached
Test suite	—	~43s wall

Suite time is dominated by three corpus walks: workflow-e2e 40s, snapshot 37s, definition-lint 37s. Untouched by this change; it is genuine measurement, not overhead.

Lean mode — the #324 D2 decision

The issue's hypothesis holds: most of the demand for a cheaper run was accidental cost, and it is gone. A full sweep is one 1.2s command; measurement no longer needs two manual runs.

Recommendation: build the challenge-pass cap if it is still wanted — it is cheap and low-risk. Keep the measurement cap deferred. Making thorough runs cheap beat making cheap runs less trustworthy, and a mode that reduces measurement is the wrong lever on a repo that just spent an issue proving its measurements were being discarded. The declared-mode shape itself is sound and would follow review mode's precedent.

Verification

• npm run typecheck — clean
• npm run check:all — 17/17, 0 unmeasured
• npm run check:delta — no new findings against origin/main
• Full vitest suite — 729 passed, 14 skipped. One local-only failure (session-crypto expects EACCES, gets ENOENT under bubblewrap); passes outside the sandbox and is unrelated to this change.

Not in scope, per the issue: reverting or reducing the guards. They are the only reason any of this was findable.

🤖 Generated with Claude Code