feat(acceptance): fail-closed fidelity gate — cold WHAT-conformance auditor + PR-body CI gate (acceptance contour Phase A) - #1102
Merged
Conversation
…ion-2 /arch phase-1)
… findings (2x REVISE, 0 BLOCKER)
…(bottom-up GO; top-down single MAJOR closed with code evidence)
…ase B choreography)
…, D1 gate-registration precondition baked into shipped texts, voluntary-BFR reframe, D6 in-session wiring
…ce requirement) + Basis delivery-seam refinement (plan review GO)
Prior-art: prior-art-evaluations.md — new entry 'PR-body fidelity acceptance gate' added in this PR (BUILD; in-repo reuse of the #1098 pr-body gate pattern). Voluntary trail: not a capability commit by the mechanical detector (~57 LOC < 80, checks/ pre-exists, no new dep).
…s shipped agents/ payload
…lity-audit MINOR)
… assertion, fail-closed bin env contract, single-block verdict semantics, mutation-killing tests - M1: workflow-integrity.yml REQUIRED_CONTEXTS now asserts fidelity-verdict-in-pr-body in staging branch protection (the gate's only fail-closed transport); workflow header reworded from present-tense claim to obligation. - M2: bin errors + exit 1 on empty BASE_REF/HEAD_SHA instead of degrading to a green no-op; ::error:: annotations; 5 new bin tests cover the env contract. - M3: exactly one Fidelity section + one FIDELITY line enforced; NON_GO checked before skipped; HTML comments stripped; H1 terminates the section; evidence may not come from the Basis line; [ \t] instead of \s; en/em/figure dashes accepted. - M4: verdict tests now use complete blocks + assert error text (verified: deleting the NON_GO branch fails 2 tests; previously 0). - Single-block invariant documented in agents/fidelity-auditor.md + PR template; new test pins the shipped template to the checker.
…gle-block invariant edit
…otection assertion is satisfiable (fidelity-audit round-2 MAJOR) GitHub reports the status-check context under the job's name:, not the job id. With a display name, workflow-integrity.yml's REQUIRED_CONTEXTS assertion could never match — and registering the id instead would deadlock every staging PR on a context that never reports. Same name==id pin as ci-success (audit-self.yml:1354).
…SEs on unchanged scope, not audits (recorded self-application violation in §7)
…itor (pre-push skill-drift gate) + baseline regen
… frontmatter (fidelity-audit round-5 MINOR — a re-execution would hit the same pre-push rejection)
…anch protection, so the required-context assertion is best-effort, not a gate Measured on PR #1102: staging protection exists with contexts ["ci-success"], yet the job printed "No branch protection or active rulesets" and exited 0. There is no workflow permission scope granting admin read (actionlint rejects `administration`), so the tri-state cannot be repaired from inside CI. The pre-existing R11 assertion has been warn-passing all along; the fidelity-context assertion inherits it. Spec §7 records the status and the two escalation paths (PAT secret / local pre-push probe), both operator decisions.
…on (fidelity-audit round-7 MAJOR) pr-body-fidelity.yml header still said the registration "is asserted" and that the assertion goes red until the operator acts — both disproved on PR #1102. The workflow-integrity warning still said "No branch protection or active rulesets" when it cannot distinguish absent from unreadable; it now reports UNDETERMINED and states plainly that its own success does not mean the contexts are registered, with the manual verify command.
…audit round-8) workflow-integrity.yml's tri-state comment still said 'no protection AND no rulesets' for an arm that is the NORMAL outcome with GITHUB_TOKEN, and asserted 'Once configured, R11 enforcement kicks in automatically' — flatly false, and the very line a maintainer reads when deciding whether the escalation paths are worth pursuing. The file admitted and denied the limitation 15 lines apart. Also: pr-body-fidelity.yml's name==id comment said workflow-integrity 'asserts' the context (it lists it), and the plan's embedded copy of the workflow header pre-dated the hardening. Comments and one doc block only — no behaviour change.
…ep-method finding The stale-claim class was patched four times by keyword grep and re-surfaced each time in new words (5th instance: 'asserting it here is what keeps that precondition a mechanism'). Changed method as the audit prescribed: read both files end-to-end and check every sentence that predicates an effect. That pass fixed the 5th instance, the two overclaiming labels (job/step names say 'check', not 'assert'), and a 6th the audit had not flagged — the KNOWN LIMITATION note quoted a warning string an earlier fix had already deleted. Header of pr-body-fidelity.yml now separates the two senses of fail-closed: it fails closed on its own subject, and is merge-blocking only once registered. Spec §7 records the method finding: a keyword grep cannot close a claim-class.
…, section boundary, D6 semantics, doc sync M2: `FIDELITY: skipped` is now rejected on a stage PR (one whose ## Provenance declares a substrate). The escape was unconditional, so the gate was opt-in for exactly the population it exists to cover, at a price of 20 chars of prose. The detector is self-declared and the spec says so — bypass now costs an explicit false statement. M6: ANY heading closes the section (was H1-H2), so the file:line evidence can no longer be satisfied by a neighbouring '### §1.7' block. Both new guards mutation-verified: reverting the boundary fails 2 tests, disabling the stage detector fails 1. M3: D6 'scope' now means the kickoff's deliverable set, never the diff — the r4 wording made every rework a reset, so the cap bounded nothing. M4: plan Global Constraints record that the forward-guard fired (57 -> 103 LOC, capability commit after all); the embedded workflow gets the name==id pin; a disclaimer marks the A2/A3 blocks as first-cut, not shipped. M5: spec §7 now records all 10 audit rounds and states plainly that the loop ended by exhaustion plus external review, not convergence. 7th stale claim: packages/preset-next-15-canonical/RULES.md said the protection check 'activates automatically once the repo moves to a paid plan or becomes public'. Measured on this PUBLIC repo with protection configured: it does not — the blocker is token scope, not billing. That file ships to consumers. Population swept: no sibling preset repeats it. Also: SSOT #228 LOC corrected; §7 method-finding no longer claims the end-to-end read closed the class (it did not — the 7th survived in a doc the diff never touched).
…ty round 11) The stage detector read only the FIRST `## Provenance` section (findIndex), so a decoy placeholder heading above the real one absorbed the check and let a stage PR use `FIDELITY: skipped` — a measured false pass. Worse, it falsified the claim shipped in two places that bypass 'requires deleting your own Provenance, a visible lie': the decoy deletes nothing and leaves the true substrate visible. Now every Provenance section is inspected, and both claims are reworded to what the code actually guarantees. Also: fenced code blocks no longer truncate a section (a `# comment` inside ```bash used to end it, failing a legitimate GO); the Basis-exclusion in the evidence check is now case-insensitive (lowercase `basis:` could smuggle the Basis path in as evidence); LOC figures in SSOT #228 and the plan corrected to the measured 165/219. Spec §7 now also applies the corrected D6 retroactively: under r5 the cap was hit at R8 (R7 REVISE → R8 REVISE, no intervening GO), which is where the session owed a STOP. 39 tests; the fix is mutation-verified (reverting to first-match-only fails 1 test).
…proposition; dispatch backward-sweep-auditor with the proposition) + remark follow-up trigger in spec §7
…lly guarantees (fidelity round-12 MINOR) The comment claimed bypass 'requires denying its own substrate'. It does not: the heading match is exact, so '## Provenance (in-session)' or '### Provenance' bypass while declaring the substrate truthfully. Both places now say what is true — every section is inspected, bypass costs a visible deviation from the shipped template, and the asymmetry is named (renaming the verdict heading fails closed; renaming Provenance fails open). Fourth instance this session of text claiming more than the code guarantees; no code change.
10 tasks
artyhoo
added a commit
that referenced
this pull request
Jul 23, 2026
…boundary + Tier-2 routing amendment (Phase B) (#1106) * feat(acceptance): /harvest §4 — fidelity verdict step before PR creation (spec D2) Prior-art: skipped — skill prose wiring only, no new capability (fidelity capability shipped PR #1102, SSOT #228). §1.7: forward-check applied — the new §4 step 2 complies with attention-is-not-a-mechanism.md §1(b) (named cold-agent detection layer, agents/fidelity-auditor.md:1) and no-paid-llm-in-ci.md §1 (session-read subagent, zero CI LLM calls); backward-check sweep — class = «skill surfaces that open a stage PR»: .claude/skills/harvest/SKILL.md:64 EXTENDED (this commit), .claude/skills/dispatcher/SKILL.md:109 GAP-FOUND (pre-egress seam, closed in the next commit of this branch), .claude/skills/night-mode/SKILL.md:33 GAP-FOUND (PR-gate sentence, closed in this branch), .claude/skills/pipeline/SKILL.md SWEPT-CLEAN (plans, never opens PRs). * feat(acceptance): /dispatcher pre-egress fidelity gate + Q&A routing seats (spec D2/D5/D6) Prior-art: skipped — skill prose wiring only, no new capability (fidelity capability shipped PR #1102, SSOT #228). * feat(acceptance): night-mode fidelity PR-gate + /arch always-marker plan-complete rule (spec D1/D2) Prior-art: skipped — skill prose wiring only, no new capability (fidelity capability shipped PR #1102, SSOT #228). * feat(acceptance): Tier-2 routing amendment — /arch plan-complete kickoffs run executor-tier pipeline (spec D1) Prior-art: skipped — routing-prose amendment only, no new capability (fidelity gate shipped PR #1102, SSOT #228). * fix(acceptance): dispatcher REVISE bullet — keep the answer.ts command on one line (list-item rendering) Prior-art: skipped — markdown rendering fix inside an existing bullet, no new capability.
This was referenced Jul 23, 2026
artyhoo
added a commit
that referenced
this pull request
Aug 10, 2026
…and gate the two lists against it (#1368) `.github/workflows/workflow-integrity.yml:75` claimed its `required_contexts` list was «the single place recording which contexts must be registered». It was neither. NOT SINGLE: `scripts/run-local-ci-sweep.sh:15-17` independently stated the same set, derived neither from it nor from the API. NOT CURRENT: live protection on staging carries three contexts (`gh api repos/artyhoo/getff/branches/staging/protection` with an admin token, 2026-08-10) — `ci-success`, `fidelity-verdict-in-pr-body`, `stale-revert-in-pr-diff` — while workflow-integrity.yml listed two. Two hand-maintained lists, one population, nothing asserting either was complete: the `#warning-nobody-reads` shape attention-is-not-a-mechanism.md §1 rejects as a detection layer. Same class as #1362, one level up. Making one list the source of truth would fix their disagreement and leave the real hole open — both could agree and both still omit a job. So the declaration moves to the population itself: every job in a `pull_request`-triggered workflow now carries a `# required-context: yes|no — <rationale>` marker, and packages/core/principles/37-required-context-completeness.test.ts asserts both lists equal the declared set exactly. audit-self.yml is carved out except for `ci-success` — its other 30 jobs route through the aggregate, which principle 36 asserts — and arm (h) asserts that sibling still exists, so the carve-out cannot silently become a hole. The substantive question — should the two unregistered discipline gates be required? — resolves differently for each, and the difference is structural, not a matter of importance: * `capability PR carries Prior-art line in PR body (squash-survival)` (pr-body-prior-art.yml) — YES. No `paths:` filter, so it reports on every PR and registering it cannot deadlock. It is in its own workflow file, so cross-file `needs:` cannot reach it and registration is its only fail-closed transport. Declared required; NOT yet registered — a live hole. * `§1.7 forward+backward sections present in PR description` (discipline-self-check.yml:32) — NO, not in its current shape. The workflow is `paths:`-filtered at the trigger, and a workflow skipped by path filtering leaves its checks Pending, blocking the PR from merging (GitHub Docs, «Troubleshooting required status checks»). Registering it would deadlock every PR touching no discipline-bearing path. Making it required needs the filter moved from `on.pull_request.paths` into the job body first — a separate change, recorded at the marker rather than left implicit. A backward sweep over the same class — «job in its own workflow file, unreachable by `ci-success.needs`» — found a third instance the report did not name: `framework-self-template-render.yml` runs the P1/P4/P6 probes on every PR, is aggregated by nothing, and is registered by nothing. Declared required, and its suite added to the local sweep (hermetic, ~5s); the alternative closure, relocating the job into audit-self.yml, is noted at the marker and deliberately not bundled. So five contexts are declared required and three are registered. Closing that delta is an operator action: CI cannot verify registration at all (workflow-integrity.yml:32-42 — GITHUB_TOKEN cannot read branch protection, measured on PR #1102), which is why that job stays best-effort and this test gates the tractable half instead. Also recorded at the list: renaming a declared-required job keeps this gate green while protection still points at the old string, so any such rename must be paired with a re-registration. Verified: principle 37 red before the markers (7/8 arms), green after; live-fired twice by mutating the real tree — deleting a marker names the exact job, and reproducing the historical one-list drift reports `declared but missing from the list: ["stale-revert-in-pr-diff"]`. Full principles suite 368 passed, actionlint clean, format:check clean, typecheck clean, run-local-ci-sweep.test.sh and meta-all-wired.test.sh both pass. Prior-art: prior-art-evaluations.md#245 (safe-settings/Terraform are registrars that apply a list without validating it against the job population — DeepWiki 2026-08-10; actionlint knows branch protection only as a webhook event; all 40 zizmor audits are security-focused. Verdict BUILD, one file, zero dependencies. Sibling of #244.) Co-authored-by: Test <test@example.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase A of the acceptance contour: the factory can now be held to what the kickoff asked for, mechanically. A cold agent protocol (
agents/fidelity-auditor.md) produces a WHAT-conformance verdict at every stage-PR boundary, and a deterministic CI gate makes that verdict's presence non-optional onbase=stagingPRs. Design altitude only — code quality stays with the existing review path. Phase B (skill/CLAUDE.md choreography wiring) ships separately.Design: docs/superpowers/specs/2026-07-23-acceptance-contour-design.md (ACCEPTED r3 after two rounds of cold two-altitude review). Plan: docs/superpowers/plans/2026-07-23-acceptance-contour.md §Phase A.
Changes
agents/fidelity-auditor.md— cold WHAT-conformance auditor: inputs are kickoff/spec + 3-dot diff only (never chat or implementation logs); reports missing/extra/diverged drift with file:line; emits the machine-consumed grammarFIDELITY: GO|REVISE|STOP+Basis/Round/Audited-SHA/Evidence;KICKOFF-AMBIGUOUSroutes a broken kickoff to re-design instead of burning rework rounds.packages/core/hooks/checks/pr-body-fidelity.ts(+.test.ts,-bin.ts) — deterministic PR-body validator:FIDELITY: GOrequires Basis + Round +Audited-SHAequal to the PR head (staleness guard: a commit pushed after the audit reddens the check) + ≥1 file:line evidence;FIDELITY: skipped — <rationale ≥20 chars>is the non-stage-PR path, and it is REJECTED on a stage PR (one whose## Provenancedeclares a substrate) — without that the gate was opt-in for exactly the population it covers. Any heading closes the section, so evidence cannot be borrowed from a neighbouring### §1.7block; fenced code blocks do not truncate it. 39 tests, key guards mutation-verified..github/workflows/pr-body-fidelity.yml— unfilteredpull_requestevents (thepr-body-prior-art.ymlfeat(prior-art): PR-body Prior-art gate — trailers survive squash (incident #1094→#1097) #1098 model; apaths:filter would un-fail-close tier-0/docs PRs), job scopedif: base.ref == 'staging'. Promote PRs (base=main) never run it; the primary exemption is registering the check as required only in the staging protection set..github/pull_request_template.md— acceptance-package sections (Provenance / Review findings / Fidelity verdict / Parked questions), placed before the §1.7 block so the §1.7 awk substance-capture boundary is unchanged.docs/meta-factory/prior-art-evaluations.md— SSOT entry 228 (BUILD).tests/install-sh/baselines/*— regenerated: the new shippedagents/*.mdfile shifts install fingerprints (SNAPSHOT_MODE=capture, then 13/13 compare green).Prior-art consult
Prior-art: prior-art-evaluations.md#228 (PR-body fidelity acceptance gate, verdict BUILD — upstream PR-body validators check form only and cannot host the semantic half, which is a session-bound cold agent under no-paid-llm-in-ci; the deterministic arm reuses the in-repo #1098 pr-body gate pattern).
Prior-art:trailer. Reclassified during implementation: the plan called this a non-capability commit at 57 LOC and recorded a forward-guard — «if the file grows ≥80 LOC, the trailer requirement becomes real». It did: closing the code-review MAJORs grewpr-body-fidelity.tsto 165 lines andpr-body-fidelity.test.tsto 219, both over the 80-LOC threshold (packages/core/hooks/checks/prior-art.ts:134). The CI gate caught the stale classification; the guard fired exactly as written.Test plan
npm test --workspace=@rules-as-tests/core --run -- pr-body-fidelity→ 39/39 (RED first: module absent, then GREEN). Mutation-checked: removing the non-GO branch fails 2 tests, reverting the section boundary fails 2, disabling the stage detector fails 1, first-match-only Provenance fails 1.npm test --workspace=@rules-as-tests/core --run -- 09-doc-authority→ 37/37 (new agent auto-enumerated, header accepted).OKrc=0; empty body →FAILrc=1 with the missing-section error;BASE_REF=main→ out-of-scope pass rc=0.actionlint .github/workflows/pr-body-fidelity.ymlclean;zizmorno findings.npm run --prefix packages/core test:principles→ 335/335;make self-auditgreen.bash scripts/run-local-ci-sweep.sh→ all gates PASS (actionlint, md-ci-only, meta-all-wired, format-check, typecheck, byte-identical, agnosticism).Provenance
origin/staging; 18 commits, 21 filesReview findings
GET /tasks/:id→plan:packages/runtime-bridge/src/cli/aifHttp.ts:15,51, consumerquestions.ts:53,89). All findings applied (spec r2/r3).old_stringanchors verified verbatim; 8 test cases traced against the regexes). Top-down REVISE → all 5 findings applied → confirmation round GO.Audited-SHA. The MAJORs that mattered:FIDELITY: skippedwas unconditional, so the gate was opt-in for exactly the population it exists to cover; the section boundary letEvidence:be satisfied by a neighbouring### §1.7block; D6's first amendment made the rework cap unfalsifiable; the plan reproduced already-fixed defects on re-execution; §7 recorded 3 audit rounds when 10 had run. A follow-up audit then found a decoy## Provenanceheading bypassing the new stage detector — closed, mutation-verified.89c33e321. The four that mattered: (1) the gate's fail-closed property rested on a manual branch-protection edit tracked only as a prose operator note —#hope-as-gateon the load-bearing precondition, now an executable assertion inworkflow-integrity.yml(REQUIRED_CONTEXTS); (2) the bin failed open — an unset/renamedBASE_REForHEAD_SHAexited 0, turning the gate into a permanently-green no-op; (3) multi-round semantics were undefined, giving both a false-pass (an appendedskippedneutralised a recorded REVISE) and a false-fail (an appended round-2 GO was shadowed by the round-1 REVISE); (4) the REVISE test was mutation-survivable — deleting the entire non-GO branch left it green. Verified after the fix: deleting that branch now fails 2 tests./archkickoffs run the executor-tier pipeline) is the first thing that genuinely depends on the registration being real. It must not ship until the registration is either mechanically verifiable (PAT secret / local pre-push probe) or converted into an explicit operator attestation. Nothing in Phase A rests on it.f55591a83); R2 REVISE on a real MAJOR — the branch-protection assertion checked the job id while GitHub reports the job'sname:, so it could never be satisfied, and registering the id would have deadlocked every staging PR on a context that never reports (fixed229983652by pinningname== id, theci-successprecedent ataudit-self.yml:1354); R4 GO, drift lists clean.The auditor re-anchored its own verdict when the head advanced mid-audit, noting that a block naming the older SHA would fail the staleness guard this PR ships — the guard proving itself on its own author.
Fidelity verdict
FIDELITY: GO
Basis: docs/superpowers/plans/2026-07-23-acceptance-contour.md#phase-a
Round: 13
Audited-SHA: 38b2bd4
Evidence: packages/core/hooks/checks/pr-body-fidelity.ts:22-29; docs/superpowers/specs/2026-07-23-acceptance-contour-design.md:52
Findings: none — drift lists missing/extra/diverged all empty
Parked questions
none
§1.7 Self-discipline check (REQUIRED if PR touches discipline-bearing files)
§1.7 Forward-check applied
This PR introduces a discipline-bearing agent + CI gate, so it is checked against every active discipline it touches. attention-is-not-a-mechanism.md §1: the detection layer is a NAMED cold-agent protocol with structured output (
agents/fidelity-auditor.md:47-57), never bare attention; the CI job is the fail-closed transport, not the judge (.github/workflows/pr-body-fidelity.yml:36-40). no-paid-llm-in-ci.md §1: the gate is pure string validation — no API-billed call anywhere in the workflow (packages/core/hooks/checks/pr-body-fidelity.ts:11-17are the only decision points); the semantic half runs session-bound under the operator's own subscription. ci-tool-pinning.md §1: both actions SHA-pinned andnpm ci --prefixused, notnpm install(.github/workflows/pr-body-fidelity.yml:26,29,33). doc-authority-hierarchy.md §2-§3: the new agent carries Class + Authoritative-for + NOT-authoritative-for (agents/fidelity-auditor.md:3-15), verified by principle 09's dynamicenumerateFlatRequiredDocs(37/37). build-first-reuse-default.md §3: BUILD verdict recorded with rationale in SSOT 228 (docs/meta-factory/prior-art-evaluations.md:301) — reuse of the in-repo #1098 gate pattern, BUILD only for the unserved semantic-half seam. language-discipline.md §1: all shipped artefacts English.§1.7 Backward-check applied
Class of this change = a PR-body section whose presence is enforced by a deterministic CI job on pull_request events. Complete enumeration of that class in the repo (
grep -l "github.event.pull_request.body" .github/workflows/*.yml+ the §1.7 arm): (1).github/workflows/pr-body-prior-art.yml:20-47— SWEPT-CLEAN: unfiltered events, no base filter, distinct section (Prior-art:line), no interaction with the new job; this PR copies its event set deliberately and does not modify it. (2).github/workflows/discipline-self-check.yml:14-24,56-66— SWEPT-CLEAN with an action taken: it ispaths:-filtered (so it is NOT the model for a fail-closed every-PR gate — that is why the new job follows #1098 instead), and its backward-check awk captures from### §1.7 Backward-check applieduntil the next/^###/; the four new template H2 sections are therefore placed BEFORE the §1.7 block (.github/pull_request_template.md:25,29,33,42all precede:50,55,58) so the capture boundary is provably unchanged. (3).github/workflows/audit-self.yml§7 real-commit trailer backstop — SWEPT-CLEAN: it is scoped tobase_ref==main && head_ref==staging(the promote flow); the new job never runs onbase=main, and its required-check registration is staging-only, so the fragile promote path (CLAUDE.mdHarness gates) is untouched. (4).github/pull_request_template.md— extended, not superseded: existing sections keep their order and content. No sibling gate is superseded, duplicated, or left inconsistent by this change.