fix(standards): generate the evaluability capture and make its guard read Core - #279
Merged
Conversation
fix: three P0s, and twelve rules that promised enforcement and delivered nothing (GT-572, GT-595, GT-604)
…read Core `native-evaluability-snapshot.json` was 12 rules behind Core and nothing said so. Its own `validation` field promised it reproduced the counts pinned by `rule-corpus-triage.spec.ts`, but the only guard over it — the test named "the evaluability snapshot still matches the class counts pinned by Core" — asserted `evaluability.counts` against six literals typed into the test file. It compared the snapshot with a second copy of the snapshot and never with Core, so it passed while the pin moved 139 -> 151 and the published handler backlog overstated the work by twelve rules. Recapture, from the live triage rather than by hand: corpus 379 -> 386 (the 7 ADR rulesets 0118..0124 were absent) native-handler 139 -> 151 documentation-only 129 -> 136 unimplemented-native 60 -> 48 The per-rule diff is closures, not a renumbering: HXA-01/02/04/05 (GT-632) and MTN-05, GIT-08, SEC-RL-01/02, ED-R04..06, DAM-R05 (GT-595) are `native-handler` now. `handlerBacklog` re-derives with them — realBacklogSize 60 -> 48, remainderToAuthor -> 38, `not-in-snapshot` back to 2 from 9. The guard is rebuilt at both ends, and neither end is a copy: - `rule-corpus-triage.spec.ts` RENDERS the snapshot from the same `CLASSIFIED` array its assertions measure, and pins it byte-for-byte — the `--check` shape `build-iso-5055-mapping.mjs` already uses for its own outputs. Recapture with UPDATE_EVALUABILITY_SNAPSHOT=1. A hand-edit now fails jest. - `iso-5055-mapping.test.mjs` parses the pinned `expect(SUMMARY.byClass)` object out of Core's suite instead of restating it. A pin it cannot find is a failure, never a skip, so the test cannot go green by losing its authority. Both were verified to go red by injecting drift from each side. Also corrects the two READMEs, which published the same stale figures in prose (60-rule backlog, "17 of 60", 381 rules), and states why the backlog moved so it does not read as a re-cut denominator. Note that `build-iso-5055-mapping.mjs --check` was already failing before this change: the corpus had grown by seven rulesets and the table had not been regenerated. Verified: node --test src/rulesets/standards/iso-5055-mapping.test.mjs (7/7), npx jest src/application/validators/rule-corpus-triage.spec.ts (22/22), build-iso-5055-mapping.mjs --check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
beyondnetPeru
added a commit
that referenced
this pull request
Jul 29, 2026
…rsion (GT-634)
Third and last version-pinned artifact the 1.2.2 bump moves.
`generate-product-inventory.mjs` stamps `@beyondnet/evolith-cli@<version>` into
`product/products/smart-cli/product-inventory.{md,es.md}`, and
`11-validate-product-docs.mjs` runs its `--check`, so `Validate documentation`
went red at "Validate product documentation synchronization". Regenerated: the
diff is the version and the counts the generator recomputes.
The staleness was reported with an EMPTY detail — `Product inventory is stale: `
— because the guard interpolates the child's stdout and the child writes its
reason to stderr. Not fixed here (it is not a release change) but worth naming:
a guard that says something is stale without saying how is a guard you have to
re-derive by hand.
WHAT I DELIBERATELY DID NOT COMMIT. `coverage-dashboard.mjs --check` also reports
drift locally, and regenerating it replaces MEASURED coverage — "88.09% statements
· 88.38% lines" — with "pending a coverage run", because this machine has no
coverage data. That is a regression dressed as a fix. The docs job on the runner
runs a build and the exploration suite before that check and passed green on
develop at 20:56Z, so the report is current where it is measured. Same for
`inventory-summary.{md,es.md}`, whose 167 -> 174 ruleset count belongs to whoever
landed #279, not to this release.
Verified with only this staged: 11-validate-product-docs OK (CLI 1.2.2, no
placeholders), 10-validate-contract-conformance OK (4 schemas), 09-reconcile
--check OK.
Honest note on method: I ran the docs job's fifteen validators in one local loop
to find every pin at once instead of one CI round each. Three of them reported
failure inside that loop and pass individually, including under the same output
redirection — so the loop, not the checks, is what was wrong. I have not explained
it, and the runner is the oracle for this job, not my loop.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Jul 29, 2026
fix(deps)!: the CLI and MCP must resolve the SDK that carries the security wave, 1.2.2 (GT-634)
#285
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Summary
src/rulesets/standards/native-evaluability-snapshot.jsonwas 12 rules behind Core, and the guard that should have caught it structurally could not.The snapshot's own
validationfield promised it reproduced the class counts pinned byrule-corpus-triage.spec.ts. The only guard over it — the test literally named "the evaluability snapshot still matches the class counts pinned by Core" — assertedevaluability.countsagainst six literals typed into the test file. It compared the snapshot with a second copy of the snapshot and never with Core, so it stayed green while the pin moved 139 → 151 and the published handler backlog overstated the remaining work by twelve rules.1. Recaptured from the live triage
classes)native-handlerdocumentation-onlyunimplemented-nativeThe per-rule diff is real closures, not a renumbering:
HXA-01/02/04/05(GT-632) andMTN-05,GIT-08,SEC-RL-01/02,ED-R04..06,DAM-R05(GT-595) arenative-handlernow. The +7 on the corpus is the ADR rulesets0118..0124, which were absent from the capture entirely.handlerBacklogre-derives with them:realBacklogSize60 → 48,remainderToAuthor→ 38,not-in-snapshotback to 2 (was 9).2. The guard now reads Core — neither end is a copy
rule-corpus-triage.spec.tsrenders the snapshot from the sameCLASSIFIEDarray its assertions measure, and pins it byte-for-byte — the--checkshapebuild-iso-5055-mapping.mjsalready uses for its own outputs. A hand-edit now fails jest. Recapture withUPDATE_EVALUABILITY_SNAPSHOT=1.iso-5055-mapping.test.mjsparses the pinnedexpect(SUMMARY.byClass)object out of Core's suite instead of restating it. A pin it cannot find is a failure, never a skip — the test cannot go green by losing its authority.Both were verified to go red by injecting drift from each side (moving Core's pin; hand-editing the snapshot).
Also in this PR
Both READMEs published the same stale figures in prose (60-rule backlog, "17 of 60", 381 rules) — corrected, with the reason the backlog moved stated so it does not read as a re-cut denominator.
Evolith Core Quality Gates
README.mdandREADME.es.mdupdated with the same figures.node .harness/scripts/ci/01-validate-docs.mjs— passed (1491 files).node .harness/scripts/ci/04-check-bilingual-parity.mjs— passed (524 docs checked).Verification
Linked ADRs / Issues
Conventional Commits
fix:).🤖 Generated with Claude Code