feat(ci): fail when a security fix is committed and unpublished, closing GT-624 - #288
Merged
Merged
Conversation
…ing GT-624
GT-624's first criterion (deprecate 1.1.0) was the small half. The wave of
2026-07-23 sat unpublished until 2026-07-27 while SECURITY.md declared the 1.1.x
line "actively patched" — four days, on a public registry, with the CHANGELOG
naming the vulnerable files. Nothing detected it; an audit did. This is the gate.
`48-validate-security-publish-lag` asks the REGISTRY what is published, finds the
commit where each package's published version was SET, and fails on any later
commit whose type or scope marks it as security.
THE REGISTRY, NOT A TAG, and that is measured rather than assumed: the newest `v*`
tag here is `v1.1.0` while npm serves `@beyondnet/evolith-cli@1.2.2`. Three
releases shipped untagged, so a git-tag oracle would have reported a four-release
lag that does not exist and flagged everything since 1.1.0.
TWO FALSE NEGATIVES, found while building it, each pinned by a fixture:
1. `git log -S'"version": "<published>"'` matches the commit that DELETED the
string as well as the one that added it. On a just-bumped package the
boundary landed on the bump itself, skipping the window.
2. Taking the NEWEST commit at the published version let a later manifest edit
that keeps the version — a dependency bump — push the boundary forward and
hide everything before it.
The boundary is now the commit where the version was SET. Denominator: 47 commits
examined across 8 packages, against 31 for the first implementation — 16 commits
the first version would have skipped in silence.
The marker is STRUCTURAL — type `security(...)`, or scope `security`/`sec` — and
refuses prose deliberately. `fix(deps)!: … the SDK that carries the security wave`
is a dependency fix; a gate that fires on any subject mentioning security gets
switched off. A unit test pins that exact subject as NOT a marker. The guard also
reports the `security(...)` type separately, because per GT-623 it is not a
Conventional Commits type and release-please derives no bump from it — a commit
that announces itself as security and cannot move a version is GT-570's failure
mode by another route.
Criterion (c) in this repository's strong form: `43-validate-guard-negative-
fixtures` OBSERVED it refuse the empty fixture, 37/37 (was 36). Registered in
`guard-classification.mjs`, so `42` counts it — 60 guards classified, 40/40
scanners refuse a zero-element scan.
Also recorded on the board rather than tidied away: two packages were deliberately
NOT deprecated. `evolith-sdk@1.1.0` is what the published line actually RESOLVED
(GT-634, fixed in 1.2.2) and `evolith-contracts@1.1.0` is that package's only
version, so deprecating it offers no upgrade path.
Verified: 12/12 fixtures over real git repositories; guard green on this repo;
08-validate-tracking (632 gaps, 575 closure records), 41 at 40 dead against a
budget of 40, 04 bilingual parity, 01-validate-docs 1491 files, 46 chain at a
fixed point. Counters 592/632 -> 593/632, pending 22 -> 21.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Summary
Closes GT-624 by building the half that mattered. Deprecating 1.1.0 (done 2026-07-29, six packages) labels what is already out. It says nothing about a fix still sitting in
main— which is exactly what happened: the 2026-07-23 security wave sat unpublished until 2026-07-27 whileSECURITY.mddeclared the 1.1.x line "actively patched". Four days, on a public registry, with the CHANGELOG naming the vulnerable files. Nothing detected it; an audit did.48-validate-security-publish-lagasks the registry what is published, finds the commit where each package's published version was set, and fails on any later commit whose type or scope marks it as security.The registry, not a tag — measured, not assumed
The newest
v*tag in this repository isv1.1.0; npm serves@beyondnet/evolith-cli@1.2.2. Three releases shipped untagged. A git-tag oracle would have reported a four-release lag that does not exist and flagged every commit since 1.1.0 as unpublished security work.Two false negatives, found while building it, each pinned by a fixture
git log -S'"version": "<published>"'matches the commit that deleted the string as well as the one that added it. On a just-bumped package the boundary landed on the bump itself, skipping the window entirely.The boundary is now the commit where the version was set. The denominator moved accordingly: 47 commits examined across 8 packages, against 31 for the first implementation — 16 commits the first version would have skipped in silence.
The marker is structural, and refuses prose on purpose
Type
security(...), or scopesecurity/sec. Not a subject grep:fix(deps)!: … the SDK that carries the security waveis a dependency fix, and a gate that fires on anything discussing security gets switched off. A unit test pins that exact subject as not a marker.The guard also reports the
security(...)type separately, because per GT-623 it is not a Conventional Commits type and release-please derives no bump from it — a commit that announces itself as a security change and cannot move a version is GT-570's failure mode by another route.Criterion (c) in this repository's strong form
Not "it ships with a fixture" but observed red:
43-validate-guard-negative-fixtures→ 37/37 exercised guards refused the empty fixture, up from 36. Registered inguard-classification.mjs, so42counts it — 60 guards classified, 40/40 scanners refuse a zero-element scan.What the gate does not do, stated rather than discovered later
It cannot detect a security fix nobody marked — that is a convention problem and belongs to GT-623. It reports, but does not fail on, a package whose published version is absent from this history (an inexact boundary after a rewrite), scanning conservatively instead of skipping silently. It needs the network: a package the registry cannot answer for is skipped and named, and a run where every package is skipped is a failure, not a pass.
Also recorded on the board rather than tidied away
Two packages were deliberately not deprecated:
evolith-sdk@1.1.0is what the published line actually resolved (GT-634, fixed in 1.2.2), andevolith-contracts@1.1.0is that package's only published version, so deprecating it offers no upgrade path.Verified
node --test 48-validate-security-publish-lag.test.mjs→ 12/12, over real git repositories in a sandbox rather than a mock of git archaeology43→ 37/37 observed red ·42→ 60 classified, 40/40 scanners08-validate-tracking→ 632 gaps, 608/608 EN/ES sections, 575 closure records41-validate-evidence-commands --strict --max-dead 40→ 40 dead, at budget: this closure record adds none04-check-bilingual-parity,01-validate-docs(1491 files),46chain at a fixed pointEvolith Core Quality Gates
01-validate-docs.mjsgreen (it caught a dangling#gt-634anchor, since that row lives on another branch; the link was removed rather than the section invented).04-check-bilingual-parity.mjsgreen.Linked ADRs / Issues
Conventional Commits
🤖 Generated with Claude Code