Skip to content

Ref hygiene: the tolerance list expires with the defect it forgives (#639) - #651

Merged
bgmcmullen merged 2 commits into
masterfrom
fix/issue-639
Aug 6, 2026
Merged

Ref hygiene: the tolerance list expires with the defect it forgives (#639)#651
bgmcmullen merged 2 commits into
masterfrom
fix/issue-639

Conversation

@philcunliffe

Copy link
Copy Markdown
Contributor

Issue #639 asked for eleven broken @ref anchors to be repaired. None of the eleven are broken. Every anchor it names resolves against origin/master today, and the repo's own gate (test/core/llp-ref-hygiene.test.js) agrees: with the tolerance list emptied it finds 0 broken annotations out of 1901.

Every ref the issue names

Ref Sites Anchor status Resolution
LLP 0103#cli src/core/commands/policy.js x5 Live: - **CLI** {#cli}: at llp/0103-machine-local-policy-classes.decision.md:63 No change. The gloss ("store, resolver and class lattice unchanged; only the verb spelling is new") is exactly what that bullet settles. Anchor was added when PR #461 landed.
LLP 0103#reporting src/core/commands/policy.js x1 Live: - **Reporting names the governing source** {#reporting}: at 0103:88 No change. The ref sits on the report path and the bullet is the requirement that the report name dotfile vs machine-local. Same PR #461.
LLP 0138#one-command src/core/cli/core_commands.js:429 Live: 0138:57 No change. The bullet is what settles that hyp agents install is removed rather than aliased, which is the one thing the skills install descriptor cannot say for itself.
LLP 0138#one-materializer src/core/cli/walkthrough.js:1390 Live: 0138:50 No change. Resolves fine, as the issue suspected it might.
LLP 0101#deadline src/core/cli/wizard/index.js:599 Live: 0101:43 No change. [constrained-by] on narration that must not invent its own deadline; the bullet owns the rule.
LLP 0101#no-release src/core/commands/sync.js:41 Live: 0101:65 No change. The bullet is the amended "release only by confirmed, attended request" that runSync implements.
LLP 0107#reversal src/core/commands/central.js:507 Live: 0107:71 No change. The bullet is what makes hyp leave responsible for org-installed assets.

All six of the non-0103 anchors are {#slug} markers on list items, and have been since the documents were written (#308, #399). A resolver that indexes only heading slugs reports every one of them as broken. .claude/skills/ref-check/SKILL.md already measures that exact failure mode at 47 references on this corpus, and the CI gate handles it correctly. The 0103 pair were genuinely broken once and were fixed by PR #461 (issue #457), which is an ancestor of master.

What was actually stale

The TOLERATED_BROKEN list in the hygiene gate. Its five entries forgave 16 occurrences; every one of them forgave nothing. PR #461 landed the repairs and the entries stayed behind, so the last surviving claim that policy.js cites a dead anchor was the list that exists to excuse it. Issue #639 read that claim back out as six live defects.

This PR:

  • Empties the tolerance list. The gate now has no exceptions to read past. Verified by running the resolver over the whole corpus with the list already empty: 0 broken.
  • Makes the next entry expire on its own. The spend ledger belongs to the caller, and a new test fails any tolerance that no longer forgives anything. Confirmed to fail when the old policy.js entry is reinserted. A stale tolerance is invisible otherwise: the suite is green either way, which is precisely how this one survived long enough to be filed as a bug.
  • Leaves the settled LLP text untouched. No anchor needed adding and no doc needed rewriting.

Checks

  • node scripts/run-tests.js: 3575 pass, 0 fail, 1 skipped (11 in the ref-hygiene file, up from 10).
  • npx tsc -p tsconfig.json --noEmit: clean.

Fixes #639

🤖 Generated with Claude Code

test and others added 2 commits August 6, 2026 03:10
…639)

Issue #639 reported eleven broken `@ref` anchors. None of them are broken.
Every anchor the issue names resolves on `origin/master`: LLP 0103 grew
`{#cli}` and `{#reporting}` when PR #461 landed, and `0138#one-command`,
`0138#one-materializer`, `0101#deadline`, `0101#no-release` and
`0107#reversal` have been `{#slug}` markers on list items since the
documents were written. A resolver that indexes only heading slugs reports
all six of those as broken, which is the failure `ref-check`'s SKILL.md
already warns is worth 47 references on this corpus.

What did survive is the tolerance list in the hygiene gate. Its five
entries, 16 forgiven occurrences, forgive nothing: PR #461 landed the
repairs and the entries stayed behind, so the only remaining claim that
`policy.js` cites a dead anchor was the list that exists to excuse it.
Issue #639 read that claim back out.

Prune the entries, and make the next one expire on its own: the spend
ledger belongs to the caller now, and a tolerance nobody spends fails the
suite instead of sitting there as evidence of a defect that healed.

With the list empty the gate has no exceptions: 1901 annotations, zero
broken.

Fixes #639

Co-Authored-By: Claude <noreply@anthropic.com>
The staleness test only checked spent > 0, so a tolerance whose budget
outran the number of genuinely broken refs at that identity passed
silently, leaving unspent forgiveness that could absorb new breakage
at the same file+target. Tighten it to spent < budget, and report both
numbers in the failure message so the offending entry is directly
editable.
@philcunliffe

Copy link
Copy Markdown
Contributor Author

Neutral review round 1 - d855f0f

This PR modifies the CI gate itself, so it was reviewed against a high bar: a weakened gate would be worse than the stale data it replaces. It is not weakened; it is strictly stronger than master. One LOW tightened in 3aa16c5.

Every check survived the refactor. All ten of master's checks were enumerated and confirmed present: the scan-reach guard, ref resolution (LLP number, anchor, docPath), the tolerance ceiling, the em-dash ban including gloss continuation lines, gloss continuation parsing, duplicate LLP numbers, and the three ignore-marker rules plus marked-file confinement. The move into brokenRefs was proven verbatim: diffing the old loop body against the new helper yields exactly one difference, the assertion that correctly stayed in the test.

Mutation results, 13 cases:

mutation result
nonexistent LLP number caught
nonexistent anchor on a real LLP caught
nonexistent docPath caught
em dash on the @ref line caught
em dash on a gloss continuation line caught
reinsert the dead policy.js entry caught by the new staleness test
reinsert all five original dead entries caught, all 5 reported
genuine break + matching budget passes (no false positive)
two breaks, budget 1 caught (ceiling survives)
suppression marker outside MARKED_FILES caught
duplicate LLP claimant caught
empty gloss / trivial gloss not caught, and not caught on master either (no such rule has ever existed)

The eleven-anchor claim independently confirmed. All seven distinct anchors exist as {#slug} markers on list items rather than headings, which is exactly the misread. The two LLP 0103 ones were genuinely broken once and fixed by PR #461 (b5fe563, merged 2026-07-30), confirming the other half of the premise. Instrumenting the gate's own extractor: 1901 refs scanned, 0 broken with tolerances forced empty.

Scope confirmed clean: one file changed, +52/-19. No @ref, LLP, or source file was quietly edited to make the gate pass.

Finding, fixed in 3aa16c5

LOW - the new staleness check tested spent > 0 rather than spent === budget. A tolerance with budget 5 against which only 1 ref is still broken would pass, leaving 4 units of forgiveness that outlived their defect and would silently absorb 4 new breakages at the same file+target. That is the realistic decay path, and it is precisely the shape that produced the stale list this PR is cleaning up: the historical entries carried counts of 5 and 6.

Tightened to (spent.get(id) ?? 0) < budget with a message reporting both numbers. Mutation-verified: an over-budget tolerance now fails (tolerance forgives 3 but only 1 are broken), an exactly-matching tolerance still passes so a legitimately needed one is not false-flagged, and the dead-entry case still fails.

npm test 3576 tests / 3575 pass / 0 fail / 1 skip, tsc clean.

@philcunliffe

Copy link
Copy Markdown
Contributor Author

Neutral review round 2 - 3aa16c5 - CLEAN

Round-1's finding is fixed. The predicate is now (spent.get(id) ?? 0) < budget and TOLERATED_BROKEN is empty.

Re-verified by mutation rather than reading, using genuine broken @refs in a scratch file, reverted after each run:

setup expected result
budget 1, 1 real break (exact match) passes passes, no false flag
budget 2, 1 real break (the round-1 hole) staleness fails fails: tolerance forgives 2 but only 1 are broken
budget 3, nothing broken (fully dead) staleness fails fails: ...but only 0 are broken, so ?? 0 handles undefined cleanly
budget 1, 2 real breaks ceiling fails fails, ceiling still enforced
nonexistent LLP number gate fails fails, LLP 9999 does not exist
nonexistent anchor on a real LLP gate fails fails, names the anchor
em dash on the @ref line fails fails, offender reported
em dash on a gloss continuation line fails fails, joined gloss shows the line

So the tightening closes the decay path without false-flagging a legitimately needed exemption, and nothing else in the gate regressed.

Edge cases probed: a zero or negative budget is inert (it can never forgive anything) and is not flagged, which is harmless. A fractional budget such as 1.5 behaves like 2 under the used >= budget comparison and is not flagged as over-provisioned, so nothing validates that budgets are non-negative integers. With the map now empty and any future entry spelled out by a human in a small diff, that is an observation rather than a finding.

Scope confirmed: one file, +57/-19 against the merge-base. npm test 3575 pass / 0 fail / 1 skip, tsc clean, no em dashes.

Nothing further outstanding.

@philcunliffe philcunliffe added the neutral:approved neutral reviewed this and holds it for a maintainer merge (own or adopted PR; LLP 0025/0030) label Aug 6, 2026
@bgmcmullen
bgmcmullen merged commit 9febbee into master Aug 6, 2026
9 checks passed
@bgmcmullen
bgmcmullen deleted the fix/issue-639 branch August 6, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

neutral:approved neutral reviewed this and holds it for a maintainer merge (own or adopted PR; LLP 0025/0030)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix the eleven pre-existing broken @ref anchors

2 participants