fix(spec): scan planned and experimental evidence, not live alone - #13081
Merged
Conversation
The liveness gate's four evidence checks (existence, line bound, symbol anchor, key mention) ran under `status === 'live'` while `producer` was scanned at any status. An entry whose whole content is a REFUSAL therefore carried evidence the census COUNTED and no check READ. Measured on api.json: `inputMapping.transform` and `outputMapping.transform` are `planned` and were migrated to `path#symbol` anchors precisely because the refusal disappearing is what should go red -- and renaming `mappingDeclarationRejection` moved no verdict. `dead` stays out, now as a declared exclusion rather than an omission: all 80 dead rows carry a `note` and only 6 carry `evidence`, so a dead row's pointer lives in prose no check scans by design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LpRNHxWZgSUgVnFT9mQQo4
Contributor
📓 Docs Drift CheckNothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs. What this run could not see
Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): |
This was referenced Aug 29, 2026
os-trump
marked this pull request as ready for review
August 29, 2026 04:11
os-trump
enabled auto-merge
August 29, 2026 04:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #13041
Disposition option 1 as ruled at dispatch (claim comment 5460051202): extend the evidence scan to
plannedandexperimental. Option 2 (the exclusion written down as a commented, argued exemption) is done alongside it, because the triage said it is worth doing whichever way option 1 went.The population change
packages/spec/scripts/liveness/check-liveness.mts, the one condition that gated all four evidence checks (existence, line bound, symbol anchor, key mention):with the partition declared, argued and held to the published vocabulary:
(Both
Setconstructions carry an explicit string type parameter in the source; it is elided here because an angle-bracket fragment does not survive this body's sanitizer, in a fence or out of one.)producerwas already scanned at any status; that asymmetry is what made the gap visible. The status vocabulary is exactly four — read fromSTATUS_COLUMNSinreadme-table.mtsand confirmed against the ledgers, which use no fifth value — so three are in and one is out, with no third category to decide about.The exhaustiveness loop is a deliberate, named addition rather than a rider: a fifth status added later would otherwise fall out of both sets and land silently unscanned, which is this card refiled. It compares two code constants, so it can only fire when someone edits
STATUS_COLUMNS.The output label is derived from the set (
EVIDENCE_SCANNED_LABEL), so the numbers and the population they describe cannot drift apart — the class of bug this whole file is about.Why
deadstays out, measuredAcross every ledger at this commit: 80
deadentries, all 80 carry anote, only 6 carry anevidencestring. A dead row's pointer is the retirement story innote— which sweep, which ADR, which tombstone rejects the key now — and no check scansnote, by design. Scanningdead.evidencewould hold 6 rows to a standard, read nothing of the other 74, and publish that as coverage of the class: the same shape as the defect being fixed, one status over. Measured for completeness: those 6 carry 3 repo-local paths and all 3 resolve today, so the exclusion is not hiding a red.Survey first — the explosion radius, before the switch was flipped
Counted before the code changed, per the triage comment's dispatch prerequisite.
Population: live 819 · planned 10 · dead 80 · experimental 5 (914 classified). Of the 15 newly-in-scope entries, 7 carry an
evidencestring at all:api/inputMapping.transformpackages/runtime/src/api-mapping.ts#mappingDeclarationRejectiontransform(8 sites)api/outputMapping.transformagent/lifecycleno runtime reader (StateMachine)agent/memoryno runtime readeragent/guardrailsno runtime readeragent/structuredOutputno runtime readertool/outputSchemapackages/services/service-ai/src/tools/action-tools.ts:437packages/services/service-ai/is a declaredFOREIGN_PATH_PREFIXESentry, so it counts as foreign and is never resolved or line-bounded. The file is genuinely absent here; that is the boundary working, not a rotThe other 8 (
action/onSuccess.navigate,action/onSuccess.openIn,field/useGrouping,field/relatedListFilter,object/externalSharingModel,page/requires,translation/flows.label,translation/flows.screens) carry noevidencekey, so they add nothing to any counter.Newly flagged: zero. No mechanical repair was needed and none was made — no ledger byte changed in this PR. The prediction from the survey was written before the switch was flipped and the run below matched it exactly, counter for counter.
Counter movement (the gate's own output, before → after)
+2 local paths and +2 anchors are the two
plannedapi.jsonrefusal pointers — the ones the card measured as "counted by the census, verified by nothing". +1 foreign istool/outputSchema. Line citations do not move: the only newly-scannedpath:NNNcitation is the foreign one, which is never bounded.key-mention.baseline.jsonneeded no new row — the two new pairs anchor.byStatusis untouched by this change, soliveness/state-counts.mdand the README table do not move.Ablation — the new red direction is real
Implementation committed first; the widened condition then reverted to the pre-fix
status === 'live'form and the gate's own test suite read on the mutated tree. Absolute paths,traprestore.On-disk confirmation, both directions (anchored at the text being replaced, not a bare
--stat):The mutated blob under which the run below was taken is the same
3b9d59a7hash, so the reading and the byte proof are about one tree.No build/dist leg, stated rather than skipped: the gate runs from source via
tsxand the test spawns that same source file, so there is nodist/artefact a stale build could mask.Prediction, written before the run: exactly 3 RED — the three new population cases — and the
dead-boundary case and the partition pin stay GREEN, because the mutation touches neither thedeadexclusion nor the two Sets.Observed (
pnpm exec vitest run scripts/liveness/check-liveness.test.tson the mutated tree,Tests 3 failed | 29 passed (32)):stays GREEN when a dead entry carries the SAME rotted pointeranddeclares every status either scanned or explicitly unscannedboth passed on the mutated tree, as predicted — so the three reds are attributable to the population change and to nothing else about the fixtures.Honest note on the transcript: the first ablation pass also printed two
grep -clines that read0in both directions — a shell-quoting error in the harness, not a reading. They were vacuous and are not quoted as evidence; the byte proof above is a clean second pass at the same blob hashes, and the first pass's real proofs (an anchor count that refused any hit-count but 1, andMUT_BLOB != HEAD_BLOB) stand on their own.Tests
check-liveness.test.ts— newdescribe('check:liveness — the evidence-scan population (#13041)'): the card's own instance (aplannedentry whose anchored refuser is renamed away must red), aplannedmissing-file case, anexperimentalmissing-file case, thedeadboundary green on the same rotted string, and a source pin on both Sets plus the published label. Four existing assertions moved off the hardcoded'live'wording onto oneSCANNED_LABELconstant.evidence.test.ts— the shipped-ledger contract test's population genuinely moves: itsstatus !== 'live'filter mirrored the gate's, and leaving it narrow would have re-created the counted-but-unverified split one layer down. The anchor contract test beside it already visited every status and is unchanged.key-mention.mts— prescription wording, which said "aliveentry" and now covers the refusal statuses.Verification
All exit codes captured before any pipe; verdicts quoted from each gate's own output. Union re-run after the final commit, at
348b3d1f1.pnpm --filter @objectstack/spec check:liveness— exit 0,✓ every governed-type property … every 'live' / 'planned' / 'experimental' entry's repo-local evidence path resolves …pnpm --filter @objectstack/spec exec vitest run --maxWorkers=2 scripts/liveness/— exit 0,Test Files 10 passed (10) · Tests 258 passed (258)tsc --noEmit -p tsconfig.scripts.json --listFiles— exit 0,0lines matchingerror TS; coverage proven rather than assumed: all four edited files appear in the program's--listFilesoutput (1 hit each), the two.test.tsfiles includedpnpm --filter @objectstack/spec check:test-typecheck— exit 0,55 file(s) / 263 error(s) held in test-typecheck-debt.json— unchanged, so this diff drifts the debt ledger up by nothingpnpm check:nul-bytes— exit 0,scanned 7275 text file(s) … no raw ASCII control bytes; plus a direct control-byte scan of the four edited files (no hits)Gate family derived with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack(26 path-derived families, plus the convention triggers for a test-file edit and a gate-script edit). Run locally:node scripts/pm/bare-root-worklist.mjs --self-testOK self-test: 47 live row(s) … none stale, none missing, none contradictedpnpm check:pm-dispatch-gates✓ dispatch-gates self-test: 872 cases pass.pnpm check:engine-double-contractOK — 704 pinned, 134 in the DEBT ledger, 3 exemptpnpm check:where-matcher313 matcher(s) discovered … 0 silently-wrong … none newpnpm check:query-options-erasureratchet holds: 67 unswept non-test site(s) … none newpnpm check:cross-package-test-inputsOK: 23 package(s) read outside themselves, all declaredpnpm check:type-check-coveragepnpm --filter @objectstack/spec check:empty-state✓ all classified (1 closed, 2 open, 4 output, 9 scope)pnpm check:published-files69 publishable package(s) … admits no test, test-harness config or build scriptDeclared narrowing: the repo-wide scans in the derived list —
pnpm lint, the docs-drift pair, the CI shard/attestation gates andcheck:type-check-debt --re-measure(which refuses an unbuilt worktree and needs the whole workspace closure built) — were not run locally. CI runs the farm exactly once regardless. The one with a real chance of moving is the test-typecheck ratchet, and its per-package half for the only package this diff touches is thecheck:test-typecheckrun above, green with the ledger count unchanged.Changeset — measured, not assumed
packages/spec'sfilesarray is["dist", "json-schema", "liveness", "prompts", "llms.txt", "README.md", "src/**/*.zod.ts", "CHANGELOG.md", "api-surface", "spec-changes.json"].scriptsis not in it, andcheck:published-filesindependently reports that the whitelists admit "no test, test-harness config or build script". This diff is entirely insidepackages/spec/scripts/liveness/— four files, none of which reach a consumer'snode_modules— and no ledger byte under the shippingliveness/directory changed. Nothing is released by this PR, so it takes theskip-changesetlabel route rather than apatchchangeset.Honest residual
liveentries (the [finding] A liveness citation can rot IN RANGE:action.targetandaction.requiredPermissionspoint at the wrong lines one day after being repointed, and every check stays green #12516 residual, unchanged in kind).statusvalues are free text: nothing validates them, andfoldStateCountsreads only the four named statuses, so a typo'd status would be classified and counted by the walk yet dropped fromstate-counts.md, and — after this change — would also fall outside both scan sets. The code-level exhaustiveness loop added here closes the code half only. The data half is out of scope for this card and filed separately (see the report on [finding] check:liveness scansevidenceonly when status islive— aplannedentry's refusal evidence is verified by nothing, measured on api.json's two transform entries #13041).Generated by Claude Code