Skip to content

fix(prior-art): the detector stops taxing test material and starts requiring a resolvable referent (#1597 ledger L-1/B-3, K-5) - #1652

Merged
artyhoo merged 4 commits into
stagingfrom
claude/strange-spence-ad630c
Sep 6, 2026
Merged

fix(prior-art): the detector stops taxing test material and starts requiring a resolvable referent (#1597 ledger L-1/B-3, K-5)#1652
artyhoo merged 4 commits into
stagingfrom
claude/strange-spence-ad630c

Conversation

@artyhoo

@artyhoo artyhoo commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

Two ways the capability gate disagreed with the definition it claims to mirror — ledger #1597 findings L-1 / B-3 (the wide half, == C13 addendum B-3) and K-5 (the real half). CLAUDE.md:30 declares the prose definition a mirror of packages/core/hooks/checks/prior-art.ts; both halves of the disagreement are closed here, in the same commit as the prose.

Changes

  • L-1 / B-3 — the LOC triggers taxed test material. packages/core/hooks/checks/prior-art.ts:231 and :249 now skip test material in both LOC arms (isExemptTestMaterial, packages/core/hooks/checks/prior-art.ts:221): *.test.* / *.spec.* files and anything under test(s)/, __tests__/ or *fixtures/. The carve-out cannot hide a capability — a commit that adds test material alongside a qualifying production file still trips on the production file — and it stops at packages/core/principles/ (ENFORCEMENT_FILE_RE, packages/core/hooks/checks/prior-art.ts:190): a principle IS the enforcement capability, not a test for one.
  • K-5 — a positive trailer needed no referent. checkTrailerBody accepted any ≥20-char payload that was not the skipped escape hatch, so Prior-art: consulted — no entry applies satisfied the gate on a capability commit. A positive line must now name something a reader can open (REFERENT_RE, packages/core/hooks/checks/prior-art.ts:74): an SSOT row (prior-art-evaluations.md#N), a concrete artefact path, or an issue/PR reference. Stacked lines keep working — a referent-free line is skipped so a later valid line can carry the commit.
  • Prose parity in the same commit. CLAUDE.md:36 now names three carve-outs on the LOC triggers instead of two, and CLAUDE.md:50 enumerates the three referent forms. Three prose↔hook sync arms pin it (packages/core/hooks/checks/prior-art.test.ts:1462).
  • Diagnostics. packages/core/hooks/pre-push.ts:383 names the three accepted referent forms in the failure message.

Measurement (why each side moved)

L-1 / B-3 — the detector is the drifted side. Over the last 250 first-parent commits on staging the ≥80-LOC arm fired on 27 commits; 18 of them added ONLY test files (9 under packages/core/principles/, 4 hook tests, 3 packages/runtime-bridge/test/, 2 skills, 1 synthesizer). The trailers it forced on the non-principle ones cited rows the commit never touched — #242/#20/#16 on a hook-test pair (the C15 case), #45 «unchanged by this» on a runtime-bridge pair. The 9 principle files, by contrast, each carried a substantive on-topic citation (#244 actionlint, #245 safe-settings, #246 Vitest test.include, #251 markdownlint, #19 lychee, #229, #230, #248) — which is why the carve-out stops there.

The split matches the other enforcement channel of the same invariant, which has held this exact semantic since it shipped: principle 11 (SSOT #48) builds its capability set from non-test files only (packages/core/principles/11-build-first-reuse-default.test.ts:192) while singling principle tests out as needing «a dedicated SSOT entry with verbatim path OR a Prior-art trailer» (packages/core/principles/11-build-first-reuse-default.test.ts:525). This PR brings the pre-push channel into parity with the CI one.

K-5 — error, not a transitional window, and a grammar wider than the strict reading. Measured over the post-cutoff (≥2026-05-12) first-parent history: 145 capability commits carry a positive Prior-art: trailer.

grammar commits it would reject rejects consulted — no entry applies?
strict #ID only (the literal documented form) 23 / 145 (16%) yes
SSOT row | artefact path | issue/PR ref (shipped) 2 / 145 (1.4%) yes

The 23 the strict reading would reject are legitimate consults naming in-repo precedent or a research patch (setup.d/lib.sh:359, research-patches/2026-05-23-guard-liveness-gate.md §2, PR #261), so the strict reading was rejected on evidence. The 2 that still fail under the shipped grammar are both from before 2026-07-19. No transitional window: the pre-push arm only inspects unpushed commits, so merged history is never re-checked, and the recent rate is 1 in the last 250 first-parent commits.

Review findings

  • MAJOR: the ≥80-LOC / ≥50-LOC triggers classed new test files as capability commits, so two thirds of the gate's firings demanded a build-vs-reuse consult for work that adds no capability, and got rote citations back. Failure-scenario: a commit whose only new packages/ file is a 100-line packages/core/hooks/hook-emit-prelude.test.ts covering an existing prelude — detectCapabilityReason returned "new file ≥80 LOC under packages/" against the pre-fix module (probe output quoted under Test plan) and the real-git repro tests/hooks/prior-art-trailer-hook.test.sh sub-test 11 exited non-zero before the fix; both are clean after it, while sub-test 12 (a new principle file) stays non-zero on both sides.
  • MAJOR: a positive Prior-art: trailer that named nothing satisfied the gate on a capability commit. Failure-scenario: a commit adding the explicit dependency some-new-dep: ^1.0.0 with the body line Prior-art: consulted — no entry appliescheckTrailerBody(...).code === 0 against the pre-fix module and the real-git repro sub-test 13 exited 0, so a genuinely new dependency merged with a trailer carrying no referent to any register row, artefact, or PR; the squash message then propagated it as the project's own precedent for what a consult looks like.

§1.7 Forward-check applied

Both fixes land on the channels the gate already owns — the pre-push §7 arm (packages/core/hooks/pre-push.ts:367 runs the same detector) and the PR-body arm (packages/core/hooks/checks/pr-body-prior-art-bin.ts:23) — with no new channel introduced, per the earliest-reachable-channel invariant. The prose↔hook obligation that CLAUDE.md:30 asserts is itself mechanised rather than left to attention: packages/core/hooks/checks/prior-art.test.ts:1462 reads CLAUDE.md from disk and fails if the capability definition stops naming the test-material carve-out or the packages/core/principles/ exception, and if the trailer-syntax section stops naming all three referent forms — the counter to #warning-nobody-reads in .claude/rules/attention-is-not-a-mechanism.md:29. Each sync arm ships with a paired negative (packages/core/hooks/checks/prior-art.test.ts:1476).

§1.7 Backward-check applied

Swept the sibling surfaces of both classes. The LOC arms had exactly two call sites, both converted (packages/core/hooks/checks/prior-art.ts:231, packages/core/hooks/checks/prior-art.ts:249); the third detector arm (isNewDepAdded) is path-independent and unaffected. The parallel enforcement channel for the same invariant was checked before choosing the carve-out shape and already excluded test files (packages/core/principles/11-build-first-reuse-default.test.ts:192), so this narrows a divergence rather than creating one. On the trailer side, the three existing tests that encoded the pre-K-5 grammar were found and updated in place rather than deleted — the two space-stripping mutation-killers keep their exact length arithmetic with #1271 standing in for filler (packages/core/hooks/checks/prior-art.test.ts:577, packages/core/hooks/checks/prior-art.test.ts:1051), and the C1 free-form case now asserts the honest post-fix semantic: a non-SSOT referent passes the existence arm because it has nothing to resolve (packages/core/hooks/checks/prior-art.test.ts:807). The two sibling PR-body gates (packages/core/hooks/checks/pr-body-fidelity.ts, packages/core/hooks/checks/pr-stale-revert.ts) parse their own trailers and share no code with this grammar. No static import was added from this shipped check to an unshipped util — the change is two module-local regexes.

Fidelity verdict

FIDELITY: skipped — no kickoff or spec substrate; this is a direct ledger-driven fix to the prior-art gate, verified by paired negatives that are RED against the pre-fix module and by a measurement over the repository's own first-parent history.

Test plan

  • npx vitest run hooks/ (in packages/core) — 1399 passed / 73 files, including the 26 new arms in packages/core/hooks/checks/prior-art.test.ts (test-material carve-out across 9 path shapes, four paired negatives, the referent grammar with its vacuity probes and stacked-line cases, three prose↔hook sync arms).
  • npx vitest run principles/ --no-file-parallelism (in packages/core) — 477 passed / 2 skipped / 47 files.
  • bash tests/hooks/prior-art-trailer-hook.test.sh19 pass / 0 fail after the fix (4 new real-git sub-tests). Before it: 17 pass / 2 fail, the failures being exactly new sub-tests 11 and 13; sub-tests 12 and 14 (the paired negative and the paired positive) are green on both sides, so the carve-out is narrowed and the grammar widened only where intended.
  • RED evidence against the pre-fix module (git show HEAD:packages/core/hooks/checks/prior-art.ts copy, run under tsx):
    • detectCapabilityReason(test-only commit) = "new file ≥80 LOC under packages/"null after
    • detectCapabilityReason(principles/99-new.test.ts) = "new file ≥80 LOC under packages/" on both sides (control)
    • detectCapabilityReason(hooks/checks/newcheck.ts) = "new file ≥80 LOC under packages/" on both sides (control)
    • checkTrailerBody('Prior-art: consulted — no entry applies').code = 01 after
    • the SSOT-row, artefact-path and PR-reference trailers = 0 on both sides (controls)
  • npx tsc --noEmit -p packages/core — clean.
  • bash scripts/build-getff-dist.sh --check — «in sync with the repo root (1064 files)»; MANIFEST regenerated in the same commit as the files it pins, and re-verified after the merge-forward.
  • SNAPSHOT_MODE=capture bash tests/install-sh/snapshot.sh — three cells move in each of the eight npm baselines (packages/core/hooks/checks/prior-art.ts, packages/core/hooks/pre-push.ts, .ai-factory/refresh-baseline.json); cargo / python / go untouched, no host-drift hunk to discard this time. Re-run after the merge-forward: no further change.
  • Pre-push hook green on the pushed head.

Prior-art consult

Prior-art: prior-art-evaluations.md#56 (§7 Prior-art trailer parser — capability-commit detection + trailer validation, verdict BUILD; this change is a semantics fix inside that row's own module, no new capability and no new dependency).

Prior-art: prior-art-evaluations.md#23 (commitlint, verdict ADOPT VOCABULARY — «custom trailer enforcement via regex on raw commit body»; the referent grammar stays inside that adopted vocabulary, adding no dependency, and the row's revisit trigger «§9 hand-roll grows beyond 2 substance arms» is unchanged: this is the §7 arm, not §9).

Not a capability change: every touched path is a modification — no new file under packages/, no new explicit dependency.

Test added 3 commits September 6, 2026 12:46
…quiring a resolvable referent (#1597 ledger L-1/B-3, K-5)

Two ways the capability gate disagreed with the definition it claims to mirror.

L-1 / B-3 — the LOC triggers counted test material. CLAUDE.md's capability
definition has always closed with «test additions for existing capabilities …
NOT capability commits», and its section header declares the prose a mirror of
this detector. Measured over the last 250 first-parent commits on staging, the
>=80-LOC arm fired on 27 commits and 18 of them added ONLY test files; the
trailers it forced cited SSOT rows the commit never touched. Test material —
*.test.* / *.spec.*, and anything under test(s)/, __tests__/ or *fixtures/ —
is now carved out of both LOC arms. A commit adding test material alongside a
qualifying production file still trips on the production file, and files
directly in packages/core/principles/ are excluded from the carve-out: a
principle IS the enforcement capability, and all 9 that fired in the same
window carried substantive on-topic citations. Both halves match the other
enforcement channel of the same invariant, which has held this semantic since
it shipped — principle 11 builds its capability set from non-test files only
(packages/core/principles/11-build-first-reuse-default.test.ts:192) while
requiring principle tests to carry «a dedicated SSOT entry with verbatim path
OR a Prior-art trailer» (:525).

K-5 — a positive trailer needed no referent. checkTrailerBody accepted any
>=20-char payload that was not the `skipped` escape hatch, so
`Prior-art: consulted — no entry applies` satisfied the gate on a capability
commit. A positive trailer must now name something a reader can open: an SSOT
row (prior-art-evaluations.md#N), a concrete artefact path, or an issue/PR
reference. The grammar is deliberately wider than the strict reading of the
documented form: over the post-cutoff first-parent history 145 capability
commits carry a positive trailer, 23 of them (16%) reference in-repo precedent
or a research patch rather than a register row. Under this grammar the same
corpus fails 2 (1.4%), both from before 2026-07-19 — hence an error, not a
transitional window. Stacked lines still work: a referent-free line is skipped
so a later valid line can carry the commit.

CLAUDE.md moves with the hook in this commit — the LOC-trigger carve-out
paragraph now names three carve-outs, and the trailer-syntax section enumerates
the three referent forms.

Prior-art: prior-art-evaluations.md#56 (§7 Prior-art trailer parser, verdict BUILD — this is that row's own module; the change is a semantics fix inside it, no new capability).
… detector change

Three cells move in each of the eight npm baselines — packages/core/hooks/checks/prior-art.ts,
packages/core/hooks/pre-push.ts, and the .ai-factory/refresh-baseline.json that records their
shipped hashes. The cargo / python / go baselines are untouched.

Prior-art: skipped — generated-artefact recapture after a code change, no new capability and no design decision to consult prior art about
@artyhoo
artyhoo enabled auto-merge (squash) September 6, 2026 09:53
…nce-ad630c

# Conflicts:
#	packages/getff/MANIFEST.sha256
#	tests/install-sh/baselines/react-native/brownfield.fingerprint
#	tests/install-sh/baselines/react-native/greenfield.fingerprint
#	tests/install-sh/baselines/react-next/brownfield.fingerprint
#	tests/install-sh/baselines/react-next/greenfield.fingerprint
#	tests/install-sh/baselines/react-spa/brownfield.fingerprint
#	tests/install-sh/baselines/react-spa/greenfield.fingerprint
#	tests/install-sh/baselines/ts-server/brownfield.fingerprint
#	tests/install-sh/baselines/ts-server/greenfield.fingerprint
@artyhoo
artyhoo merged commit 473a555 into staging Sep 6, 2026
47 checks passed
@artyhoo
artyhoo deleted the claude/strange-spence-ad630c branch September 6, 2026 10:30
artyhoo pushed a commit that referenced this pull request Sep 6, 2026
…ferent grammar and the test-material carve-out

Found by this PR's own backward check, which swept every surface that teaches an
author what a `Prior-art:` line may look like. CONTRIBUTING.md:3 declares itself
authoritative for «capability-commit definition + Prior-art trailer convention»,
so it is a teaching surface for both halves of #1652 — and it was stale on both:
its syntax block still carried «narrative referencing prior-art-evaluations.md#<ID>,
or escape hatch», and its trigger table listed the two LOC thresholds with none
of the detector's carve-outs.

Both are now stated: the three accepted referent forms with a worked example
each, and the three LOC carve-outs including test material with its
packages/core/principles/ exception. A sync arm reads the file from disk and
fails if either enumeration stops being named.

Prior-art: prior-art-evaluations.md#56 (§7 Prior-art trailer parser, verdict BUILD — the contributor-facing documentation of that row's own convention; a doc-parity fix, no new capability and no new dependency).
artyhoo added a commit that referenced this pull request Sep 6, 2026
…nt grammar (#1597 ledger K-5 follow-up) (#1655)

## Summary

Follow-up to #1652 (ledger #1597 **K-5**), same concern, coordinator-ACKed. That PR widened the positive-trailer grammar from «cite `prior-art-evaluations.md#N`» to «name a resolvable referent» — an SSOT row, a concrete artefact path, or an issue/PR reference — on the measurement that **23 of 145** capability commits in the post-cutoff first-parent history rest on in-repo precedent or a research patch rather than a register row. Two teaching surfaces were left describing the pre-fix version of that grammar: the stderr hint the PR-body gate prints, and the CONTRIBUTING.md section that declares itself authoritative for the convention.

The second one was **not in the ACKed scope** — it was found by this PR's own backward check, which swept every surface that tells an author what a `Prior-art:` line may look like. Aligning it here rather than deferring it: it is the same concern (a teaching surface contradicting the gate), it is in the same doc family as the `CLAUDE.md` paragraph #1652 already moved, and shipping a fix for one stale twin while knowingly leaving the other is the drift this campaign exists to close.

## Changes

- **The CI hint enumerates all three forms**, with a worked example each — `packages/core/hooks/checks/pr-body-prior-art-bin.ts:44`-`:47`. It keeps the escape-hatch warning and the pointer at the CLAUDE.md section that owns the grammar.
- **CONTRIBUTING.md catches up on both halves of #1652.** Its trailer-syntax block (`CONTRIBUTING.md:207`) now states the three referent forms with an example each and names the referent-free assertion the hook rejects; its capability-commit section (`CONTRIBUTING.md:175`) now states the detector's three LOC carve-outs — docs, byte-identical relocations, and test material with its `packages/core/principles/` exception — where before it listed the two thresholds with no carve-out at all.
- **Both are pinned by sync arms** — `packages/core/hooks/checks/pr-body-prior-art.test.ts:190` reads the bin from disk, `:210` reads CONTRIBUTING.md, and each fails if an enumeration stops being named or if the «cite an SSOT entry instead» phrasing returns. Paired negative built from the pre-fix hint text.

**Scope note (deliberate non-change):** the pre-push substance-arm messages at `packages/core/hooks/pre-push.ts:415` and `packages/core/hooks/pre-push.ts:425` stay as they are. They fire when an author took `Prior-art: skipped` on a capability commit, where citing the register IS the right advice — they are not the «which forms are accepted» hint, so widening them would blur two different diagnostics.

## Review findings

- MINOR: the PR-body gate's own error hint named only one of the three referent forms it accepts. Failure-scenario: a capability PR whose body carries `Prior-art: REUSE — setup.d/lib.sh:359 (the copy_safe idiom this reuses)` fails some *other* check, the author re-reads the prior-art gate's hint while fixing, and «cite an SSOT entry instead» tells them their valid trailer is wrong — pushing a spurious append to `docs/meta-factory/prior-art-evaluations.md`, which is an append-only register per its §3. The sync arm is RED against the pre-fix hint (`2 failed | 15 passed`, quoted under Test plan) and green after.
- MINOR: `CONTRIBUTING.md:3` claims authority over «capability-commit definition + Prior-art trailer convention» while both of its enumerations were stale. Failure-scenario: a contributor follows the documented path — reads CONTRIBUTING, writes `Prior-art: <narrative referencing prior-art-evaluations.md#<ID>>` for a commit whose only new packages/ file is a 90-line `*.test.ts`, and is told by the doc to consult prior art the gate no longer asks for; conversely a reader of its trigger table concludes a new test file IS a capability commit, which the shipped detector contradicts. The CONTRIBUTING sync arm is RED against the pre-fix file (`1 failed | 17 passed`) and green after.

### §1.7 Forward-check applied

No new channel: this is the diagnostic text of an existing CI gate (`.github/workflows/pr-body-prior-art.yml` → `packages/core/hooks/checks/pr-body-prior-art-bin.ts:23`) plus the contributor doc that mirrors it. What is new is that neither is maintained by attention any more — `packages/core/hooks/checks/pr-body-prior-art.test.ts:190` and `:210` read both files and assert each accepted form is still named, the same mechanised-parity shape #1652 applied to the CLAUDE.md prose, and the counter to `#warning-nobody-reads` in `.claude/rules/attention-is-not-a-mechanism.md:29`. Each arm ships with a paired negative so it cannot pass vacuously.

### §1.7 Backward-check applied

Swept every surface that tells an author what a `Prior-art:` line may look like — five exist. `CLAUDE.md:50` (the ratified grammar) and the pre-push §7 failure message (`packages/core/hooks/pre-push.ts:383`) were updated in #1652. The pre-push substance-arm messages (`packages/core/hooks/pre-push.ts:415`, `packages/core/hooks/pre-push.ts:425`) are deliberately unchanged — see the scope note. This PR closes the remaining two: the CI bin hint and `CONTRIBUTING.md:207` / `:175`. `.github/pull_request_template.md:11` was checked and carries no enumeration — it links to CONTRIBUTING rather than restating the forms, so it inherits the fix instead of drifting. The grammar itself is untouched: `REFERENT_RE` at `packages/core/hooks/checks/prior-art.ts:74` remains the single definition, and this PR changes no gate behaviour — only what the gate prints and what the docs claim.

The first draft of this section asserted CONTRIBUTING.md carried no enumeration; running the grep instead of trusting the recollection is what surfaced it.

## Fidelity verdict

FIDELITY: skipped — no kickoff or spec substrate; this is the coordinator-ACKed diagnostic-text half of the K-5 fix landed in #1652, plus the stale contributor-doc twin its backward check surfaced, each verified by a sync arm RED against the pre-fix file.

## Test plan

- `npx vitest run hooks/checks/pr-body-prior-art` (in `packages/core`) — **18 passed** after the change.
- RED-first, bin hint: the same file run against the pre-fix bin (`git show HEAD:packages/core/hooks/checks/pr-body-prior-art-bin.ts` restored in place) — **2 failed | 15 passed**, the failures being exactly «the hint names all three accepted referent forms» and «the hint no longer claims the SSOT row is the only way».
- RED-first, CONTRIBUTING: the same file run against the pre-fix `CONTRIBUTING.md` — **1 failed | 17 passed**, the failure being exactly the new CONTRIBUTING arm.
- In both RED runs the paired negative and the CLAUDE.md-pointer arm stayed green, so the arms assert the change and not the files' mere existence.
- `npx vitest run hooks/checks/` (in `packages/core`) — **415 passed**.
- `npx tsc --noEmit -p packages/core` — clean.
- `bash scripts/build-getff-dist.sh --check` — «in sync with the repo root (1066 files)»; MANIFEST regenerated in the same commit as the files it pins.
- `SNAPSHOT_MODE=capture bash tests/install-sh/snapshot.sh` — no baseline moved. Expected: the bin is a CI entrypoint and CONTRIBUTING.md is not part of the consumer-shipped payload, so neither appears in an install fingerprint.
- `wc -l CONTRIBUTING.md` — 273, well under the 600-line markdown gate.
- Pre-push hook green on the pushed head.

## Prior-art consult

Prior-art: prior-art-evaluations.md#56 (§7 Prior-art trailer parser — capability-commit detection + trailer validation, verdict BUILD; this changes that row's own CI diagnostic text and the contributor documentation of its convention, no new capability and no new dependency).

Not a capability change: all four touched paths are modifications — no new file under `packages/`, no new explicit dependency.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant