From c3a3ace1318c3d9779e0decad5401756d51ee4ca Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 08:09:05 +0000 Subject: [PATCH] feat(spec): gate the liveness README's state table against GOVERNED (#7257) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The ledger README's "Current state" table is the liveness ledger's own index — one row per governed type, counts regenerated from the gate's --json report, a hand-written Notes cell per row. Its heading, `## Current state — N governed types (complete registry coverage)`, is a completeness CLAIM that nothing could falsify: N was the count of ROWS, not of governed types, and the two agreed only by coincidence. They stopped. `api` and `capability` were both in GOVERNED, both had ledgers, both were counted by the gate, and neither had a row. Same shape the file spends 500 lines warning about, one level up: `dashboard. widgets` asserted in prose that its 22 child keys were classified in a subtree that never existed, and survived a release because prose cannot fail a build (#4956). Every other claim in the file became data the gate resolves; the index was the last one riding on a human reading it. - readme-table.mts — the fourth direction, pure and unit-tested for the same reason as orphans.mts: on a green tree the table is complete, so a passing gate proves nothing about whether the check can fire. Fails (not warns) on a missing row, an orphan row, a duplicated row, or a heading N that disagrees with either the row count or GOVERNED.length. All three heading legs, because two of them agreeing is exactly the state this issue found. - No new check:/gen: script — it rides inside check:liveness, which the Spec property liveness workflow already runs on packages/spec/**. check:generated --reconcile-only stays green with no ledger change. - The documented regeneration snippet now reads the table back as well as the report, so a governed type with no row prints a SKELETON row: the omission is visible at regeneration time as well as at CI time. - The two missing rows are back-filled from their seeding PRs' own measurements — counts from --json, prose from what #5271/PR #5312 and #5961/PR #6540 actually measured. The Notes cell is the one thing the gate deliberately does not check, which is why they were filed rather than fixed on the spot. - Also removes a stray blank line that had been splitting the table in two. Reverse verification is pinned in CI, not just performed once: three cases in check-liveness.test.ts drive the REAL gate against a copy of the real README with a row deleted, the heading skewed, and an ungoverned row added, and read exit 1. Closes #7257 --- .changeset/liveness-readme-table-gated.md | 48 ++++ packages/spec/liveness/README.md | 40 ++- .../spec/scripts/liveness/check-liveness.mts | 83 ++++++- .../scripts/liveness/check-liveness.test.ts | 71 ++++++ .../spec/scripts/liveness/readme-table.mts | 227 ++++++++++++++++++ .../scripts/liveness/readme-table.test.ts | 205 ++++++++++++++++ 6 files changed, 663 insertions(+), 11 deletions(-) create mode 100644 .changeset/liveness-readme-table-gated.md create mode 100644 packages/spec/scripts/liveness/readme-table.mts create mode 100644 packages/spec/scripts/liveness/readme-table.test.ts diff --git a/.changeset/liveness-readme-table-gated.md b/.changeset/liveness-readme-table-gated.md new file mode 100644 index 0000000000..55fb2ec883 --- /dev/null +++ b/.changeset/liveness-readme-table-gated.md @@ -0,0 +1,48 @@ +--- +"@objectstack/spec": patch +--- + +`check:liveness` reconciles the ledger README's "Current state" table against `GOVERNED`, +and the two rows it was missing (`api`, `capability`) are back-filled. + +That table is the liveness ledger's own index — one row per governed type, counts +regenerated from the gate's `--json` report, and a hand-written Notes cell recording how +each type got where it is. It opens with a heading of the form +`## Current state — N governed types (complete registry coverage)`, which is a completeness +CLAIM, and nothing could falsify it: `N` was the count of ROWS, not of governed types, so +the two agreed only by coincidence. They stopped. `api` (seeded 2026-08-04, #5271/#5206, +PR #5312) and `capability` (seeded 2026-08-08, #5961, PR #6540) were both in `GOVERNED`, +both had ledgers, both were counted by the gate, and neither had a row — under a heading +that still read as true to every subsequent reader (#7257). + +This is the shape the file spends 500 lines warning about, one level up. `dashboard.widgets` +asserted in prose that its 22 child keys were "classified in the DashboardWidgetSchema +subtree" — a subtree that never existed — and survived a release because **prose cannot fail +a build** (#4956). Every other claim in that file has since become data the gate resolves: +schema → ledger, ledger → schema, container → declared disposition, `GOVERNED` → the +metadata-type registry in both directions. The index was the last one riding on a human +reading it. + +So it becomes the gate's fourth direction (`scripts/liveness/readme-table.mts`, pure and +unit-tested for the same reason as `orphans.mts`: on a green tree the table is complete, so +a passing run proves nothing about whether the check can fire). It FAILS — not warns — when +a `GOVERNED` type has no row, when a row exists that `GOVERNED` does not back, when a type is +claimed by two rows, or when `N` disagrees with either the row count or `GOVERNED.length`. +All three heading legs are checked, because two of them agreeing is exactly the state #7257 +found. No new `check:`/`gen:` script: it rides inside `check:liveness`, which the +`Spec property liveness` workflow already runs on every PR touching `packages/spec/**`. + +The documented regeneration snippet now reads the table back as well as the report, so a +governed type with no row prints a **skeleton row** instead of silently not being printed +next to its siblings — the omission surfaces at regeneration time as well as at CI time. +Both were needed: the count columns get regenerated far more often than the row set gets +audited. + +What the gate deliberately does not check is the Notes cell, which is hand-written +measurement — a manufactured one is worse than a missing row, and that is why the two rows +were filed rather than fixed on the spot. Both are back-filled here from their seeding PRs' +own measurements: counts from `--json`, prose from what #5271/#5312 and #5961/#6540 actually +measured. Both types turn out to be the same worked example — **enforced but undeclared**, +the mirror of this ledger's usual `declared ≠ enforced`: each was already being consumed at +runtime while absent from the metadata-type registry, so `saveMetaItem` stored arbitrary JSON +against it. Docs and tooling only; no runtime behaviour changes. diff --git a/packages/spec/liveness/README.md b/packages/spec/liveness/README.md index 1006ee74a0..2cbff08f4f 100644 --- a/packages/spec/liveness/README.md +++ b/packages/spec/liveness/README.md @@ -647,12 +647,21 @@ The governed set is `GOVERNED` at the top of `check-liveness.mts`. To add a type ## Current state — 30 governed types (complete registry coverage) -> The table below carries 28 of the 30. `api` and `capability` are governed -> (they are in `GOVERNED`, they have ledgers, the gate counts them) and were -> added without a row here — the table fell behind its own registry, which is -> the shape this file keeps warning about one level down. Filed rather than -> back-filled from a guess: writing two Notes cells for changes somebody else -> measured is exactly the fabrication the drill section forbids. +> **This heading is now checked** (#7257). `check:liveness` reconciles the table +> against `GOVERNED` in both directions — a governed type with no row fails, a row +> no `GOVERNED` entry backs fails, and `N` must equal both the row count *and* +> `GOVERNED.length`. Until that landed the number was the count of ROWS, not of +> governed types, and the two agreed only by coincidence: `api` and `capability` +> were governed, ledgered and counted by the gate for days with no row here, under +> a heading that still said coverage was complete. Prose cannot fail a build — the +> same sentence this file spends 500 lines making true one level down (#4956), and +> the last claim in it that was still riding on a human reading the file. +> +> The two missing rows were back-filled in the same change, from the seeding PRs' +> own measurements rather than from the counts — see `api` and `capability` at the +> foot of the table. What the gate deliberately does **not** check is the part +> that made #7257 worth filing rather than fixing on the spot: the Notes cell is +> hand-written measurement, and a manufactured one is worse than a missing row. **The counting method for this table is the gate's own report** — `check-liveness.mts --json`, `types..byStatus` — decided in #4488 after @@ -669,13 +678,25 @@ count columns are **never hand-edited** — regenerate: ```bash cd packages/spec && npx tsx scripts/liveness/check-liveness.mts --json | python3 -c " -import json,sys +import json,re,sys r = json.load(sys.stdin) +have = set(re.findall(r'^\|\s*([a-z][a-z0-9_]*)\s*\|', open('liveness/README.md').read().split('## Current state')[-1], re.M)) +NEW = ' **NO ROW YET (#7257) — write this Notes cell from the seeding PR measurement, never from a guess** |' for t, v in r['types'].items(): b = v['byStatus'] - print(f\"| {t} | {b.get('live',0)} | {b.get('experimental',0)} | {b.get('dead',0)} | {b.get('planned',0)} |\")" + row = f\"| {t} | {b.get('live',0)} | {b.get('experimental',0)} | {b.get('dead',0)} | {b.get('planned',0)} |\" + print(row + ('' if t in have else NEW))" ``` +The snippet reads the table back as well as the report, so **a governed type with +no row prints a skeleton** instead of silently not being printed next to 29 +siblings that are. That is the omission made visible at REGENERATION time; the +gate above makes it visible at CI time. Both were needed: `api` and `capability` +survived because the only reader who could have noticed was a human comparing two +lists by eye, and the count columns get regenerated far more often than the row +set gets audited. The skeleton stops at the count columns on purpose — it prints +a marker where the Notes cell goes, never a guess at what belongs there. + | Type | live | exp | dead | planned | Notes | |---|---|---|---|---|---| | object | 49 | – | 0 | 1 | aspirational tier (versioning/softDelete/search/recordName/keyPrefix) + tags/active/abstract REMOVED (#2377) — tombstoned in UNKNOWN_KEY_GUIDANCE; `enable.trash`/`mru` REMOVED (#2377 close-out) — tombstoned in the now-`.strict()` ObjectCapabilities; `isSystem` + `enable.searchable` CORRECTED to live (#2377 — sharing default-model + global-search opt-out; 2026-06 audit missed both readers); `tenancy.strategy`/`crossTenantAccess` REMOVED post-15.0 (#2763) | @@ -691,7 +712,6 @@ for t, v in r['types'].items(): | dataset | 27 | – | 0 | – | `measures.certified` (declared-but-unenforced governance flag) REMOVED in 16.0 (#2377) | | page | 16 | – | – | 1 | fully live + one planned | | view | 79 | 0 | 4 | – | list/form drilled via `children` (#2998 Track B); list.{responsive,performance} + form.{defaultSort,aria} REMOVED 2026-07-30 (#3896 close-out sweep — list aria/data stay live); **form.data was that sweep's one CORRECTION** — the removal attempt broke the build (`defineForm` writes `data.provider='schema'` onto every metadata form, `metadata-protocol` serves it), so it stands `live` with re-verified evidence; form.{buttons,defaults} live (framework#1894 / #2998); audit-era DEAD lines superseded by re-verification; level-2 dead residue (userActions.buttons, addRecord.mode/formView, tabs[].order) noted on parents — one drill level only | - | report | 21 | 0 | 0 | – | dataset-bound (ADR-0021); the aria/performance LEDGER entries were stale — the keys left the schema in the report-liveness close-out; deleted 2026-07-30 as hygiene. Audit-era `chart` DEAD superseded (framework#1890 / #3441) | | dashboard | 34 | 0 | 7 | – | ADR-0021 dataset widgets (#3251; DashboardWidgetSchema `.strict()`); `aria`/`performance` (and widget `performance` + PerformanceConfigSchema) REMOVED 2026-07-30 (#3896 close-out sweep — no renderer applied any of them); audit-era `globalFilters`/`dateRange` DEAD superseded (framework#2501) | **#4956**: `widgets` DRILLED — the row jumps 20 → 41 classified because all 22 widget-level keys enter the count at once. They had never been classified at all: the entry carried one blanket `live` plus a `note` asserting they were classified "in the DashboardWidgetSchema subtree", and no such subtree existed in any of the 28 ledger files. That gap, not any evidence, is what carried `widgets[].responsive` through the #3896 sweep that removed both its sibling `widgets[].performance` and its literal namesake `view.responsive` — `view` is drilled, so `list.responsive` got asked and went out. New dead 6 = `responsive` (retired #4876/#4995, tombstone keeps the row) + `colorVariant` + `actionUrl`/`actionType`/`actionIcon` + `aria`. The action trio is the sharpest: no renderer draws a per-widget action button at all (every `actionUrl` read in DashboardRenderer is scoped to `header.actions[]`), yet `validate-dashboard-action-refs.ts` enforces reference integrity on it and its docblock calls it "the per-widget button" — a lint guarding an affordance that does not exist. `requiresService` is the counter-example worth remembering: dead by every objectui measurement, and LIVE server-side (`filterDashboardForUser`, ADR-0057 D10) — judging a widget key from the renderer repo alone would have retired an enforced gate. `compareTo` is `live` on ONE path only (inline object-provider charts); on the ADR-0021 dataset path the string arms are dropped and `{ offset }` throws in the executor. **#6774** moves the row 33/8 → 34/7: `colorVariant` CORRECTED dead → live 2026-08-09, the enforce leg of #5010 ruling B landing from the renderer side (objectui#3359 / PR objectui#3799, absorbed by pin `09987b68`). Worth reading beside `requiresService` above, because it is the same lesson from the other end — that row warns against judging a widget key from the renderer repo alone, and this one is a `dead` verdict that was correct in this repo AND correct in the renderer repo on the day it was measured, and stopped being either when a cross-repo decision was implemented. A ledger row is a claim with a timestamp; `verifiedAt` is what makes the claim re-askable. It also empties the dashboard warn set, so the author-side lint now says nothing about any widget key — `dashboard` stays in the lint's TYPE_COLLECTIONS all the same (the `webhook`/`email_template` resolved state) | | query | 15 | 1 | 5 | 0 | **not a metadata type** — the REQUEST surface (`QuerySchema`: client SDK QueryBuilder output; the `POST /data/:object/query` body), governed via `SPEC_ONLY_SCHEMAS` (#4286). The gate's one-level walk resolves 1 experimental; the 7 marker-experimental search affordances sit one level deeper, below the walk — resolved from `[EXPERIMENTAL — not enforced]` describe markers, not ledger entries (search `fuzzy`/`operator`/`boost`/`minScore`/`language`/`highlight` + `aggregations[].filter` — declared engine affordances no executor receives). The #4286 sweep closed out same-release: `having` ENFORCED 2026-07-31 (engine-side post-aggregation filter, both paths; was finding 1); dead 4 = the tombstoned removals `joins`/`windowFunctions`/`cursor`/`distinct` — REMOVED 2026-07-31 (retiredKey keeps each in the walked shape so the rows stay; protocol-17 semantic migrations; the JoinNode + WindowFunctionNode clusters and the `QueryBuilder.cursor()`/`.distinct()` producers deleted with their keys; `distinct`'s mis-wired REST count suppression deleted too — finding 2). **#6815** adds the 5th dead: `aggregations[].distinct` REMOVED 2026-08-09 (live → dead, `-1` live). It is the one member of this ledger the #4286 sweep could not have caught with the question it asked — that sweep looked for keys NO executor reads, and this one had a reader: the objectql in-memory fallback deduplicated before applying the function while all five other faces (driver-sql, driver-turso, driver-mongodb, driver-memory, service-analytics' `AGGREGATE_SQL`) ignored it, so one query answered two plausible NUMBERS depending on which backend served it. The lesson for the next audit is the question, not the key: a per-key `live` verdict is only as good as the count of faces it was measured across, and this row's 2026-07-31 evidence (`in-memory-aggregation.ts:167,204-206`) was TRUE and still the wrong verdict. `count_distinct` is the surviving spelling (enforce leg, #6409) | @@ -707,6 +727,8 @@ for t, v in r['types'].items(): | translation | 17 | – | 2 | – | seeded 2026-08-01 (#4488) — after fixing the walker: the registered schema is a z.preprocess pipe (#3778 retired-dialect guard) whose transform side the unwrap always took, so the type was literally unwalkable. 10 of 11 groups live across spec resolvers, REST localization, objectui client resolvers and plugin-audit (whose composed-key `t()` calls make `messages` easy to mis-verify as dead). Dead 1 = `validationMessages` (authorWarn): nothing resolves it, and #3778's own legacy-key migration table steers `errors:` authors into it — a shipped false signpost, the capabilities.readOnly shape | **#4667**: `validationMessages` REMOVED (row deleted) — removed from the shared translationDataShape(), so it retired at BOTH doors at once, closing the item-only asymmetry #3778's original guard had. #3778's own `errors` guidance was rewritten in the same change: it had been steering authors INTO this dead group. | | qa | 4 | – | 5 | – | seeded 2026-08-10 (#6247) — **not a metadata type**: `TestSuiteSchema` is the FILE surface of the shipped `os test` command (`qa/*.test.json`), governed through the same `SPEC_ONLY_SCHEMAS` override as `query`/`webhook`/`validation`. It is in the table as the clearest worked example of a **false `dead` measurement**: #6247 reported the whole domain declared-but-inert on a grep that scanned only `*Schema` identifiers, and every consumer here reads the **type** names (`QA.TestSuite`, `QA.TestStep`, `QA.TestAction`) — so an entire execution chain (core's `TestRunner` + `HttpTestAdapter`, published via `export * as QA`, driven by a documented CLI command) read as zero consumers, and a retire ruling was issued on it before being withdrawn. The `evidenceScope` table one section up says no amount of specifier matching is sufficient for a negative claim; this is the same lesson for **identifier** matching. What was really wrong was narrower and real: the type was the contract and the schema had no `parse` site, so the CLI's `JSON.parse(content) as QA.TestSuite` cast admitted anything — ENFORCED in the same change (`TestSuiteSchema.safeParse` at the load site, pinned). Dead 5 = `name` (the file name is the suite identity; the CLI prints `path.basename`), `scenarios.name` (describe() says "for test reports"; every report carries `scenarioId` instead), `scenarios.description` (docs-shaped, kept), and the two on the enforce-or-remove worklist — `scenarios.tags` promises filtering that `os test`'s two flags cannot express, and `scenarios.requires` declares param/plugin preconditions nothing checks, so a suite naming a missing plugin runs anyway and fails as an unexplained HTTP error. Neither carries `authorWarn` and the omission is deliberate (`_authorWarnSkipped`): the lint walks stack **collections**, a QA suite is a loose file in no stack, so a warn flag here would emit nothing — a silent no-op inside the mechanism built to catch silent no-ops | | validation | 15 | 0 | 3 | 0 | seeded 2026-08-01 (#4488). The ADR-0020 carrier: the evaluator honors active/events/priority/severity/type/condition/message (the zod header's "only reads type/condition/…" prose is STALE — trust the ledger). Dead 3 = label/description/tags, declared governance metadata, kept unmarked. Union walk boundary recorded: only base + `script` keys walked; per-variant keys are governed by the evaluator's tests, not ledger rows. **No longer a registered metadata kind** — #4509 retired it under ADR-0088 (a standalone rule had no object-binding key and every variant is `.strict()`, so it bound to nothing and gated no write; a state machine authored that way saved cleanly and did nothing). The rule VOCABULARY is untouched and fully live via `object.validations[]`, so the ledger keeps governing it through the gate's spec-only override, alongside `webhook` and `query`. The contrast with the two bridges in the same batch is the point: enforce-or-remove picked ENFORCE where the feature existed and only the wiring was missing, and REMOVE where the shape itself could not carry the feature | +| api | 25 | 0 | 0 | 2 | seeded 2026-08-04 (#5271, part of #5206; PR #5312) — **not a metadata type until that same change made it one**, which is the row's point: governance and registration landed together, the treatment `datasource` did not get (#4487) and paid for with six inert keys found by hand. What #5206 measured before the fix: `api` was in neither `DEFAULT_METADATA_TYPE_REGISTRY` nor `BUILTIN_METADATA_TYPE_SCHEMAS`, so `saveMetaItem`'s `resolveOverlaySchema('api', …)` → `getMetadataTypeSchema('api')` returned `undefined` and took its own documented branch — an unregistered type is stored **unvalidated** — while `getMetaTypes()` could not enumerate the type at all, so Studio rendered neither list nor form. That issue names the shape precisely and it is the inverse of this ledger's usual one: **enforced but undeclared** (the matcher was already indexing these entries, #5089), where `dead` is declared-but-unenforced. The seeding pass classified 27 keys — live 25 / planned 2 / dead 0 — each cited `file:line` at the consumer layer that reads it: the MATCHER (`packages/metadata/src/endpoint-matcher.ts`) indexes `name`/`path`/`method`; the EXECUTOR (`packages/runtime/src/endpoint-executor.ts`) dispatches on `type` and reads `target`/`objectParams`; the POLICY chain (`packages/runtime/src/endpoint-policy.ts` + `security/inbound-rate-limit.ts`) enforces `authRequired`/`rateLimit`/`cacheTtl`; the MAPPING layer (`packages/runtime/src/api-mapping.ts`) applies `inputMapping`/`outputMapping`; and OpenAPI enrichment (`packages/rest/src/openapi-endpoints.ts`) emits `summary`/`description`. Timing was the reason it was cheap: #5040's E-series had built every one of those consumers and all of it was on main, so each key had a real evidence path rather than a promise. **Planned 2 = `inputMapping.transform` + `outputMapping.transform`, and `planned` rather than `dead` is load-bearing**: `dead` here means parsed with no consumer — a silent no-op — and these are the opposite, parsed and then LOUDLY REFUSED at publish (`endpoint-publish-gate.ts` mappingGate) and again at runtime, because no transformation-function registry exists anywhere in the platform. An author who writes one is told so and told what to do instead, so there is nothing for enforce-or-remove to chase; they stay in the vocabulary because admitting them needs a function registry **and** a sandbox ruling (#5040 §3.4), which is a design decision, not a key to quietly delete. Zero dead | +| capability | 12 | 0 | 0 | 0 | seeded 2026-08-08 (#5961; PR #6540) — `CapabilityDeclarationSchema`, the DECLARATION side of ADR-0066 D1's three-way separation: packages DEFINE a capability, permission sets GRANT it via `systemPermissions`, resources REQUIRE it via `requiredPermissions`. **The gate's 12 and the seeding PR's 5 are the same measurement at two granularities** — PR #6540 call-graph-closed **5 authorable properties**, every one to a real reader in `packages/plugins/plugin-security/src/bootstrap-declared-capabilities.ts` (the one consumer that turns a declaration into a `sys_capability` row), all `live`, with no `PENDING_GOVERNANCE` debt recorded; the other 7 are the ADR-0010 protection-envelope keys the gate auto-classifies `live` and which carry `null` verdicts in the file, exactly as on `permission`/`position`. The same worked example as `api` above and PR #6540 says so in those words — **enforced but undeclared**, the mirror of the hole #5271 closed. What #5961 measured: absent from `DEFAULT_METADATA_TYPE_REGISTRY`, `BUILTIN_METADATA_TYPE_SCHEMAS` and `HAND_CRAFTED_SCHEMAS`, so `isRuntimeCreateAllowed()` took its no-static-entry fallback (permanently true) and `saveMetaItem` its no-schema branch — `PUT /api/v1/meta/capability/:name` accepted **arbitrary JSON** onto an authorization surface whose names `systemPermissions`/`requiredPermissions` resolve by string, while `/meta/types` synthesised a false `allowRuntimeCreate: true` descriptor Studio drew a raw-JSON create form from. #5870 did not open that path (the write gate reads the registry, not the item store); it only made the type visible in `getMetaTypes()`, and both the issue and this row say so to stop the next reader filing it as a regression. Landed as ruling A on ADR-0066 D1's own authority: `allowRuntimeCreate: false` **and** `allowOrgOverride: false`, the second self-judged inside the ruling's rationale and flagged for veto — a tenant overlay of a package declaration would lift `scope` from `org` to `platform`, which is the one field on this type that is an escalation rather than display. Its reverse verification is worth copying: deleting the registry entry gave 7 red / 3 green and measured something **sharper than predicted** — a garbage payload turned 422 rather than resolving, i.e. the schema binding is a real second line of defence behind the registry row, not a restatement of it; deleting the schema binding alone gave exactly 3 red. `packageId` is the one key that reads oddly: deliberately a FALLBACK, not the primary, since #5870 added `capabilities` to the ObjectQL stamped-collection list so `_packageId` now reaches a declaration and wins — it stays `live` because the fallback branch still decides materialization for any declaration arriving unstamped. Zero dead | The `dead` set across types is the enforce-or-remove worklist (ADR-0049); every misleading entry carries `authorWarn` so authors hear about it at compile time diff --git a/packages/spec/scripts/liveness/check-liveness.mts b/packages/spec/scripts/liveness/check-liveness.mts index eccb6ba964..cb8cfbbcba 100644 --- a/packages/spec/scripts/liveness/check-liveness.mts +++ b/packages/spec/scripts/liveness/check-liveness.mts @@ -61,6 +61,14 @@ // one and the authored value was never read. An optional `"producer"` cites the // call site; its paths resolve exactly like `evidence`. // +// THE README'S OWN INDEX (#7257): the ledger README ends with a "Current state" +// table, one row per governed type, under a heading that claims complete +// registry coverage. Nothing reconciled that table against `GOVERNED`, so the +// heading's number was the count of ROWS rather than of governed types, and +// `api` and `capability` were governed for days with no row at all. Same shape +// as #4956 one level up: a completeness sentence that no build could fail. See +// readme-table.mts. +// // EVIDENCE SCOPE (`evidenceScope`, #4895): four measured verdicts have been // reached by searching this repo alone and published as if they covered every // consumer (`app.homePageId`, `flow.…position`, `HttpMethod`, @@ -123,6 +131,12 @@ import { reconcileContainerCoverage, type ContainerCoverage, } from './drill.mts'; +import { + README_ORPHAN_ROW_GUIDANCE, + README_TABLE_GUIDANCE, + parseStateTable, + reconcileReadmeTable, +} from './readme-table.mts'; const here = dirname(fileURLToPath(import.meta.url)); const specRoot = resolve(here, '../..'); // packages/spec @@ -361,6 +375,11 @@ const report: any = { brokenDeferrals: [] as string[], // a declared deferral whose target is missing, drifted, or double-declared deferredContainers: [] as string[], // containers whose subtree IS classified elsewhere — resolved, not believed deferredChildKeys: 0, // how many child keys those resolved deferrals actually cover + readmeMissingRows: [] as string[], // a GOVERNED type with no row in the README state table (#7257) + readmeOrphanRows: [] as string[], // a README row for a type GOVERNED does not contain + readmeHeadingErrors: [] as string[], // "N governed types" disagrees with the rows / with GOVERNED + readmeMalformedRows: [] as string[], // a table line the row parser could not read — never silently skipped + readmeRowCount: 0, // rows the parser found, printed every run so the number is visible rather than believed verification: null as VerificationReport | null, // `verifiedAt` ages — the re-verification worklist producers: null as ProducerReport | null, // `producer` / `evidenceScope` — the #4837 / #4895 worklists producerMissing: [] as string[], // a `producer` pointer into thin air — FAILS, like a rotted `evidence` @@ -567,6 +586,28 @@ report.undrilledChildKeys = coverage.inheritedChildKeys; report.deferredContainers = undrilledBaseline.deferred.map((d) => `${d.container} → ${d.to}`); report.deferredChildKeys = coverage.deferredChildKeys; +// ── the README's own index: does the state table cover GOVERNED? ── +// The gate's fourth direction (#7257). The three above ask whether the LEDGERS +// are honest; this asks whether the file that indexes them is. It reads the +// README from the ledger root so the self-test can point it at a mutated copy — +// the same `--ledger-root` trick the evidence guard uses, and for the same +// reason: a gate that fails is worth exactly as much as the proof that it fails, +// and this table is complete on a green tree. +const readmeFile = join(ledgerRoot, 'README.md'); +if (!existsSync(readmeFile)) { + report.readmeHeadingErrors.push(`${readmeFile} does not exist — the ledger index is gone`); +} else { + const stateTable = parseStateTable(readFileSync(readmeFile, 'utf8')); + const readme = reconcileReadmeTable({ governed: GOVERNED, table: stateTable }); + report.readmeMissingRows = readme.missingRows; + report.readmeOrphanRows = readme.orphanRows.concat( + readme.duplicateRows.map((d) => `${d} — duplicate row`), + ); + report.readmeHeadingErrors = readme.headingErrors; + report.readmeMalformedRows = readme.malformed; + report.readmeRowCount = stateTable.rows.length; +} + // ── verifiedAt: how old is each claim? ── // Age never fails the gate — re-verification is a worklist, not a merge gate. // A MALFORMED value does fail: it silently disables the staleness check for @@ -618,7 +659,11 @@ const failed = report.stalePending.length > 0 || report.undrilledNew.length > 0 || report.undrilledStale.length > 0 || - report.brokenDeferrals.length > 0; + report.brokenDeferrals.length > 0 || + report.readmeMissingRows.length > 0 || + report.readmeOrphanRows.length > 0 || + report.readmeHeadingErrors.length > 0 || + report.readmeMalformedRows.length > 0; if (asJson) { process.stdout.write(JSON.stringify(report, null, 2) + '\n'); } else { @@ -732,6 +777,39 @@ if (asJson) { ' `containers` list and admit the keys are classified nowhere.', ); } + if (report.readmeMissingRows.length) { + console.log( + `\n✗ ${report.readmeMissingRows.length} governed type(s) with NO row in the README's ` + + '"Current state" table — the index fell behind its own registry:', + ); + report.readmeMissingRows.forEach((t: string) => console.log(` ${t}`)); + console.log(''); + README_TABLE_GUIDANCE.forEach((line) => console.log(line ? ` ${line}` : '')); + } + if (report.readmeOrphanRows.length) { + console.log(`\n✗ ${report.readmeOrphanRows.length} README state-table row(s) that GOVERNED does not back:`); + report.readmeOrphanRows.forEach((t: string) => console.log(` ${t}`)); + console.log(''); + README_ORPHAN_ROW_GUIDANCE.forEach((line) => console.log(line ? ` ${line}` : '')); + } + if (report.readmeHeadingErrors.length) { + console.log(`\n✗ ${report.readmeHeadingErrors.length} README state-table heading error(s):`); + report.readmeHeadingErrors.forEach((s: string) => console.log(` ${s}`)); + console.log( + '\n "N governed types (complete registry coverage)" is a completeness CLAIM, and\n' + + ' until #7257 nothing could falsify it: N was the count of rows, not of governed\n' + + ' types, so the two agreed only by coincidence — until `api` and `capability` were\n' + + ' governed with no row and the sentence stayed true-looking. Fix the number and the\n' + + ' rows together; they are checked against each other AND against GOVERNED.', + ); + } + if (report.readmeMalformedRows.length) { + console.log( + `\n✗ ${report.readmeMalformedRows.length} unreadable line(s) in the README state table —` + + ' a row this parser cannot see is a row it cannot govern:', + ); + report.readmeMalformedRows.forEach((s: string) => console.log(` ${s}`)); + } // ── re-verification clock ── // Annotated at the boundary: `report` is deliberately `any` (see its // declaration), so without this every `v.*` below is `any` too — which is how @@ -827,7 +905,8 @@ if (asJson) { '\n✓ every governed-type property at the walk\'s one-level granularity is classified, every ' + 'registered type is governed or explicitly pending, no ledger row outlives its property, ' + 'every container inheritance is declared, every `live` entry\'s repo-local evidence path ' + - 'resolves, and all bound high-risk proofs resolve.', + 'resolves, all bound high-risk proofs resolve, and the README state table carries a row ' + + `for each of the ${report.readmeRowCount} governed type(s) it claims to index.`, ); if (report.undrilledChildKeys) { console.log( diff --git a/packages/spec/scripts/liveness/check-liveness.test.ts b/packages/spec/scripts/liveness/check-liveness.test.ts index 4bd5efe381..61d25d80df 100644 --- a/packages/spec/scripts/liveness/check-liveness.test.ts +++ b/packages/spec/scripts/liveness/check-liveness.test.ts @@ -136,6 +136,77 @@ describe('check:liveness — evidence pointers (#5623)', () => { }); }); +// The README state table is COMPLETE on a green tree (#7257 back-filled the two +// rows that were missing), so `pnpm check:liveness` passing says nothing about +// whether this direction can fire. Same argument as the evidence guard above, +// and the same mechanism answers it: `--ledger-root` points the gate at a copy +// of packages/spec/liveness — which `cpSync` carries README.md into — so a case +// can delete a row or skew the heading in the COPY and read the real exit code. +describe('check:liveness — the README state table (#7257)', () => { + let tmp: string; + + beforeAll(() => { + tmp = mkdtempSync(path.join(tmpdir(), 'os-liveness-readme-')); + }); + afterAll(() => rmSync(tmp, { recursive: true, force: true })); + + /** Copy the real ledgers (README included) and rewrite the README in the copy. */ + function withReadme(name: string, edit: (md: string) => string): string { + const root = path.join(tmp, name); + cpSync(LEDGERS, root, { recursive: true }); + const file = path.join(root, 'README.md'); + writeFileSync(file, edit(readFileSync(file, 'utf8'))); + return root; + } + + it('FAILS when a governed type loses its row — the #7257 defect itself', () => { + // `qa` is the most recently added row (#6247 / PR #7255), so deleting it + // reproduces the exact state the two missing rows were in. + const root = withReadme('missing-row', (md) => + md.split('\n').filter((l) => !l.startsWith('| qa | ')).join('\n')); + + const { status, output } = runGate(root); + expect(status, output).toBe(1); + expect(output).toContain('governed type(s) with NO row in the README\'s "Current state" table'); + expect(output).toMatch(/^ {4}qa$/m); + // The prescription has to survive with the check: without it the next agent + // to hit this failure writes a Notes cell out of the counts, which is the + // fabrication #7257 refused to commit. + expect(output).toContain('never from a guess'); + }); + + it('FAILS when the heading count is skewed away from the rows', () => { + const root = withReadme('skewed-heading', (md) => + md.replace(/^## Current state — (\d+) governed types/m, '## Current state — 99 governed types')); + + const { status, output } = runGate(root); + expect(status, output).toBe(1); + expect(output).toContain('README state-table heading error(s)'); + expect(output).toContain('heading says 99 governed types, the table has'); + expect(output).toContain('GOVERNED has'); + }); + + it('FAILS on a row that GOVERNED does not back — the mirror direction', () => { + const root = withReadme('orphan-row', (md) => + md.replace(/^\| qa \| /m, '| notatype | 1 | 0 | 0 | 0 | invented by the self-test |\n| qa | ')); + + const { status, output } = runGate(root); + expect(status, output).toBe(1); + expect(output).toContain('README state-table row(s) that GOVERNED does not back'); + expect(output).toMatch(/^ {4}notatype$/m); + }); + + // The control for all three: the same copy, unedited, is green. Without it + // every "exit 1" above is also satisfied by the copy simply being unreadable. + it('is green against a verbatim copy, and says how many rows it checked', () => { + const root = path.join(tmp, 'verbatim'); + cpSync(LEDGERS, root, { recursive: true }); + const { status, output } = runGate(root); + expect(status, output).toBe(0); + expect(output).toMatch(/README state table carries a row for each of the \d+ governed type\(s\)/); + }); +}); + describe('check:liveness — the evidence summary line (#5623)', () => { let tmp: string; diff --git a/packages/spec/scripts/liveness/readme-table.mts b/packages/spec/scripts/liveness/readme-table.mts new file mode 100644 index 0000000000..7515aa4583 --- /dev/null +++ b/packages/spec/scripts/liveness/readme-table.mts @@ -0,0 +1,227 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// The README state table — the ledger's own index, reconciled against `GOVERNED`. +// +// WHY THIS EXISTS. `packages/spec/liveness/README.md` opens its last section with +// a heading of the form `## Current state — N governed types (complete registry +// coverage)` and one row per governed type. That heading is a COMPLETENESS CLAIM, +// and until this check landed nothing could falsify it: `N` was the count of ROWS, +// not the count of governed types, and the two agreed only by coincidence. +// +// They stopped agreeing. `api` (seeded 2026-08-04, #5271/#5206) and `capability` +// (seeded 2026-08-08, #5961/PR #6540) were both added to `GOVERNED`, both given +// ledgers, both counted by the gate — and neither got a row. The heading still +// said the registry coverage was complete, because the sentence was checked by +// nothing (#7257). +// +// That is the same failure shape this README spends 500 lines warning about, one +// level up. `dashboard.widgets` asserted in prose that its 22 child keys were +// "classified in the DashboardWidgetSchema subtree" — a subtree that never +// existed — and the claim survived a release because PROSE CANNOT FAIL A BUILD +// (#4956). Every other claim in that file eventually got turned into data the +// gate resolves: schema → ledger, ledger → schema, container → declared +// disposition, `GOVERNED` → the metadata-type registry in both directions. The +// file's own index was the last claim still riding on a human reading it. +// +// So the reconciliation is a FOURTH direction, and it fails rather than warns. +// The population is small and exact (one row per governed type), there is no debt +// to amortise once the two missing rows are back-filled, and a warning here would +// re-create the original defect one layer up: this README's own verdict is that a +// permanently-noisy check is a check nobody reads. +// +// WHAT IT DOES NOT CHECK, deliberately: the count COLUMNS and the Notes cell. The +// counts are regenerated from `check-liveness.mts --json` (the method fixed in +// #4488) and the Notes cell is hand-written measurement — "how this type got where +// it is", the one part of the table a script cannot author. Holding the numbers to +// the gate is a separate, larger job than holding the ROW SET to `GOVERNED`, and +// conflating them would have made this check unlandable. Presence is the claim the +// heading makes; presence is what this resolves. + +/** One parsed row of the "Current state" table. */ +export interface StateTableRow { + /** The type named in the row's first cell. */ + type: string; + /** 1-based line number in the README — so a failure can be opened, not hunted. */ + line: number; +} + +/** The "Current state" section, as data. */ +export interface ParsedStateTable { + /** 1-based line of the `## Current state — N governed types …` heading, or `null` if absent. */ + headingLine: number | null; + /** `N` as the heading declares it, or `null` when the heading is absent / carries no count. */ + headingCount: number | null; + /** The heading text verbatim, for the failure message. */ + headingText: string | null; + /** Every row whose first cell is a type token, in file order. */ + rows: StateTableRow[]; + /** + * Table lines inside the section that are neither the header, the separator, nor + * a recognisable type row. Reported rather than skipped: a row this parser cannot + * see is a row the reconciliation cannot govern, which is the #4956 shape again. + */ + malformed: string[]; +} + +export interface ReadmeReconciliation { + /** A `GOVERNED` type with no row — the #7257 defect itself. */ + missingRows: string[]; + /** A row for a type `GOVERNED` does not contain — the mirror, an orphan row. */ + orphanRows: string[]; + /** The same type claimed by two rows; the row count would then over-state coverage. */ + duplicateRows: string[]; + /** The heading is missing, unparseable, or its `N` disagrees with the row count / `GOVERNED`. */ + headingErrors: string[]; + /** Passed through from the parse so the caller reports one population, not two. */ + malformed: string[]; +} + +const HEADING_RE = /^##\s+Current state\b/; +const HEADING_COUNT_RE = /(\d+)\s+governed types/; +const SEPARATOR_CELL_RE = /^:?-{3,}:?$/; +const TYPE_CELL_RE = /^[a-z][a-z0-9_]*$/; + +/** + * Parse the README's "Current state" section. + * + * The section runs from its own `##` heading to the next `##` heading or EOF, and + * fenced code blocks inside it are skipped — the documented regeneration snippet + * contains a `print(f"| {t} | …")` line that is a template for rows, not a row. + */ +export function parseStateTable(markdown: string): ParsedStateTable { + const lines = markdown.split('\n'); + const result: ParsedStateTable = { + headingLine: null, + headingCount: null, + headingText: null, + rows: [], + malformed: [], + }; + + let inSection = false; + let inFence = false; + + for (let i = 0; i < lines.length; i++) { + const raw = lines[i]; + const line = raw.trim(); + + if (!inSection) { + if (HEADING_RE.test(line)) { + inSection = true; + result.headingLine = i + 1; + result.headingText = line; + result.headingCount = Number(line.match(HEADING_COUNT_RE)?.[1] ?? NaN); + if (Number.isNaN(result.headingCount)) result.headingCount = null; + } + continue; + } + + if (line.startsWith('```')) { inFence = !inFence; continue; } + if (inFence) continue; + if (line.startsWith('## ')) break; // the section ended + if (!line.startsWith('|')) continue; + + const first = line.slice(1).split('|')[0].trim(); + if (first === 'Type') continue; // the header row + if (SEPARATOR_CELL_RE.test(first)) continue; // the `|---|` rule + if (TYPE_CELL_RE.test(first)) { result.rows.push({ type: first, line: i + 1 }); continue; } + result.malformed.push(`line ${i + 1}: ${line.slice(0, 80)}`); + } + + return result; +} + +/** + * Reconcile the parsed table against `GOVERNED`. + * + * Three-way on the heading, and all three legs matter for a different reason: + * `headingCount === rows.length` is the arithmetic a reader checks by eye and + * never does; `rows.length === governed.length` is the coverage claim; and + * `headingCount === governed.length` is the sentence itself. Two of the three + * agreeing is exactly the state #7257 found — the heading matched the rows, and + * both were short of the registry. + */ +export function reconcileReadmeTable({ + governed, + table, +}: { + governed: readonly string[]; + table: ParsedStateTable; +}): ReadmeReconciliation { + const rowTypes = table.rows.map((r) => r.type); + const rowSet = new Set(rowTypes); + const governedSet = new Set(governed); + + const seen = new Set(); + const duplicateRows: string[] = []; + for (const r of table.rows) { + if (seen.has(r.type)) duplicateRows.push(`${r.type} (line ${r.line})`); + seen.add(r.type); + } + + const headingErrors: string[] = []; + if (table.headingLine === null) { + headingErrors.push( + 'the "## Current state — N governed types" heading is gone — the table this ' + + 'gate reconciles is identified by it', + ); + } else if (table.headingCount === null) { + headingErrors.push( + `the heading carries no "N governed types" count: ${table.headingText}`, + ); + } else { + if (table.headingCount !== rowTypes.length) { + headingErrors.push( + `heading says ${table.headingCount} governed types, the table has ${rowTypes.length} row(s)`, + ); + } + if (table.headingCount !== governed.length) { + headingErrors.push( + `heading says ${table.headingCount} governed types, GOVERNED has ${governed.length}`, + ); + } + } + + return { + missingRows: governed.filter((t) => !rowSet.has(t)), + orphanRows: rowTypes.filter((t) => !governedSet.has(t)).sort(), + duplicateRows, + headingErrors, + malformed: table.malformed, + }; +} + +/** + * The prescription printed under a missing-row failure. It names the ONE thing a + * script cannot do for you, because that is the whole reason the two rows this + * check was written for were filed rather than back-filled (#7257). + */ +export const README_TABLE_GUIDANCE = [ + 'Every type in GOVERNED needs a row in the README\'s "Current state" table. The', + 'table is the ledger\'s index — it is what a human or an AI reads first to learn', + 'what this ledger covers — and its heading claims complete registry coverage.', + '', + 'Regenerate the count columns; never hand-edit them:', + '', + ' cd packages/spec && npx tsx scripts/liveness/check-liveness.mts --json | python3 …', + '', + '(the exact snippet is in the README, above the table; it now prints a SKELETON', + 'row for any governed type that has no row yet, so paste that row in.)', + '', + 'Then write the Notes cell BY MEASUREMENT, never from a guess. It records how', + 'this type got where it is — the seeding PR, what that PR actually measured,', + 'which keys are dead and why. Do NOT infer one from the counts or from the', + 'type\'s name, and do not write one for somebody else\'s change: a manufactured', + 'Notes cell is the drill section\'s own prohibition ("do not drill by fanning a', + 'parent\'s status out over its children; that manufactures verdicts, which is', + 'worse than the gap") applied to this table. If a Note cannot be honestly', + 'sourced, write the measured counts plus a pointer to the seeding PR and stop.', +]; + +/** The prescription for the mirror direction — a row no `GOVERNED` entry backs. */ +export const README_ORPHAN_ROW_GUIDANCE = [ + 'A row for a type that is not in GOVERNED is the same rot as an orphan ledger', + 'row, one level up: it claims coverage of something this gate does not govern,', + 'and it inflates the row count the heading is checked against. Either govern the', + 'type (add it to GOVERNED and seed its ledger) or delete the row.', +]; diff --git a/packages/spec/scripts/liveness/readme-table.test.ts b/packages/spec/scripts/liveness/readme-table.test.ts new file mode 100644 index 0000000000..1ece6e7893 --- /dev/null +++ b/packages/spec/scripts/liveness/readme-table.test.ts @@ -0,0 +1,205 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// Unit tests for the README state-table reconciliation (#7257) — the liveness +// gate's fourth direction. See readme-table.mts for why it exists. +// +// Same reasoning as orphans.test.ts and drill.test.ts, and it applies harder +// here: once the two missing rows are back-filled the table is COMPLETE, so a +// green `check:liveness` proves nothing about whether this check can fire. That +// proof comes from here (the logic, including every case it must stay quiet on) +// and from check-liveness.test.ts (the real gate, against a mutated copy of the +// real README, reaching a real `process.exit(1)`). + +import { describe, it, expect } from 'vitest'; +import { + README_ORPHAN_ROW_GUIDANCE, + README_TABLE_GUIDANCE, + parseStateTable, + reconcileReadmeTable, +} from './readme-table.mts'; + +/** A miniature README with the same section shape as the real one. */ +function readme({ + heading = '## Current state — 3 governed types (complete registry coverage)', + rows = ['| object | 49 | – | 0 | 1 | notes |', '| field | 66 | 0 | 0 | 0 | notes |', '| api | 25 | 0 | 0 | 2 | notes |'], + before = '## Adding a type\n\nSome prose.\n\n| Status | Meaning |\n|---|---|\n| `live` | Has a runtime consumer. |\n', + after = '', +}: { heading?: string; rows?: string[]; before?: string; after?: string } = {}): string { + return [ + before, + heading, + '', + '| Type | live | exp | dead | planned | Notes |', + '|---|---|---|---|---|---|', + ...rows, + after, + ].join('\n'); +} + +describe('parseStateTable', () => { + it('reads the heading count, the heading line and one row per type', () => { + const t = parseStateTable(readme()); + expect(t.headingCount).toBe(3); + expect(t.headingLine).toBeGreaterThan(0); + expect(t.rows.map((r) => r.type)).toEqual(['object', 'field', 'api']); + expect(t.malformed).toEqual([]); + }); + + it('ignores the header row and the |---| rule', () => { + const t = parseStateTable(readme()); + expect(t.rows.map((r) => r.type)).not.toContain('Type'); + expect(t.rows).toHaveLength(3); + }); + + // The boundary that decides whether the parser sees the table or the file. A + // `## Status vocabulary` table above the section is table-shaped and its rows + // are NOT governed types; counting them would inflate the row count and make + // the heading check fire on a correct file. + it('only reads rows inside the "Current state" section', () => { + const t = parseStateTable(readme()); + expect(t.rows.map((r) => r.type)).not.toContain('live'); + }); + + it('stops at the next ## heading', () => { + const t = parseStateTable(readme({ after: '\n## Something else\n\n| ghost | 1 | 0 | 0 | 0 |\n' })); + expect(t.rows.map((r) => r.type)).not.toContain('ghost'); + }); + + // The documented regeneration snippet contains `print(f"| {t} | …")`, which is + // a TEMPLATE for rows. Reading it as a row would make the gate fail on the + // instructions for fixing it. + it('skips fenced code blocks — the regeneration snippet is not a row', () => { + const t = parseStateTable( + readme({ after: '\n```bash\nfor t in x: print(f"| {t} | 1 | 0 | 0 | 0 |")\n```\n' }), + ); + expect(t.rows).toHaveLength(3); + expect(t.malformed).toEqual([]); + }); + + it('reports a table line it cannot read rather than skipping it', () => { + const t = parseStateTable(readme({ rows: ['| object | 49 | – | 0 | 1 | notes |', '| ??? | 1 | 0 | 0 | 0 |'] })); + expect(t.rows.map((r) => r.type)).toEqual(['object']); + expect(t.malformed).toHaveLength(1); + expect(t.malformed[0]).toContain('???'); + }); + + it('records a missing heading as absent rather than throwing', () => { + const t = parseStateTable('# Spec liveness ledger\n\nNo state section at all.\n'); + expect(t.headingLine).toBeNull(); + expect(t.headingCount).toBeNull(); + expect(t.rows).toEqual([]); + }); + + it('records a heading that carries no count', () => { + const t = parseStateTable(readme({ heading: '## Current state' })); + expect(t.headingLine).toBeGreaterThan(0); + expect(t.headingCount).toBeNull(); + }); +}); + +describe('reconcileReadmeTable — what it must catch', () => { + const governed = ['object', 'field', 'api']; + + it('is quiet when every governed type has a row and the heading agrees', () => { + const r = reconcileReadmeTable({ governed, table: parseStateTable(readme()) }); + expect(r).toEqual({ + missingRows: [], + orphanRows: [], + duplicateRows: [], + headingErrors: [], + malformed: [], + }); + }); + + // #7257 itself: the type is governed, ledgered and counted by the gate, and + // the index that claims to cover it does not. + it('catches a governed type with no row', () => { + const table = parseStateTable( + readme({ heading: '## Current state — 2 governed types (complete registry coverage)', rows: ['| object | 49 | – | 0 | 1 | n |', '| field | 66 | 0 | 0 | 0 | n |'] }), + ); + const r = reconcileReadmeTable({ governed, table }); + expect(r.missingRows).toEqual(['api']); + // ...and the heading's own arithmetic is intact, which is exactly how the + // real defect stayed invisible: N matched the ROWS and both were short of + // GOVERNED. Two of the three legs agreeing is not coverage. + expect(r.headingErrors).toEqual(['heading says 2 governed types, GOVERNED has 3']); + }); + + it('names every missing row, not just the first', () => { + const r = reconcileReadmeTable({ + governed: [...governed, 'capability', 'qa'], + table: parseStateTable(readme()), + }); + expect(r.missingRows).toEqual(['capability', 'qa']); + }); + + it('catches the mirror — a row GOVERNED does not back', () => { + const table = parseStateTable( + readme({ heading: '## Current state — 4 governed types (complete registry coverage)', rows: ['| object | 49 | – | 0 | 1 | n |', '| field | 66 | 0 | 0 | 0 | n |', '| api | 25 | 0 | 0 | 2 | n |', '| retired | 1 | 0 | 0 | 0 | n |'] }), + ); + const r = reconcileReadmeTable({ governed, table }); + expect(r.orphanRows).toEqual(['retired']); + expect(r.missingRows).toEqual([]); + }); + + // A duplicate would let the row count reach GOVERNED.length while a real type + // has no row — the heading check alone would pass. + it('catches a duplicated row', () => { + const table = parseStateTable( + readme({ rows: ['| object | 49 | – | 0 | 1 | n |', '| field | 66 | 0 | 0 | 0 | n |', '| field | 66 | 0 | 0 | 0 | n |'] }), + ); + const r = reconcileReadmeTable({ governed, table }); + expect(r.duplicateRows).toHaveLength(1); + expect(r.duplicateRows[0]).toContain('field'); + expect(r.missingRows).toEqual(['api']); + }); + + it('catches a heading count skewed away from the rows', () => { + const t = parseStateTable(readme({ heading: '## Current state — 30 governed types (complete registry coverage)' })); + const r = reconcileReadmeTable({ governed, table: t }); + expect(r.headingErrors).toEqual([ + 'heading says 30 governed types, the table has 3 row(s)', + 'heading says 30 governed types, GOVERNED has 3', + ]); + }); + + it('fails loudly when the heading is gone', () => { + const r = reconcileReadmeTable({ governed, table: parseStateTable('# Spec liveness ledger\n') }); + expect(r.headingErrors).toHaveLength(1); + expect(r.headingErrors[0]).toContain('heading is gone'); + // Every governed type is also missing — the failure is real, not a parse + // artefact being quietly swallowed. + expect(r.missingRows).toEqual(governed); + }); + + it('fails when the heading carries no count', () => { + const r = reconcileReadmeTable({ governed, table: parseStateTable(readme({ heading: '## Current state' })) }); + expect(r.headingErrors).toHaveLength(1); + expect(r.headingErrors[0]).toContain('no "N governed types" count'); + }); + + it('passes malformed table lines through so the caller reports one population', () => { + const table = parseStateTable(readme({ rows: [...['| object | 49 | – | 0 | 1 | n |', '| field | 66 | 0 | 0 | 0 | n |', '| api | 25 | 0 | 0 | 2 | n |'], '| Total | 140 |'] })); + const r = reconcileReadmeTable({ governed, table }); + expect(r.malformed).toHaveLength(1); + expect(r.missingRows).toEqual([]); + }); +}); + +describe('the prescriptions', () => { + // The one thing a script cannot do for the author, and the reason #7257 was + // filed rather than fixed on the spot. If this sentence ever leaves the + // guidance, the next agent to hit the failure will fabricate a Notes cell. + it('tells the author to source the Notes cell from measurement, never a guess', () => { + const text = README_TABLE_GUIDANCE.join('\n'); + expect(text).toContain('Notes cell'); + expect(text).toContain('never from a guess'); + expect(text).toContain('seeding PR'); + }); + + it('tells the author both ways out of an orphan row', () => { + const text = README_ORPHAN_ROW_GUIDANCE.join('\n'); + expect(text).toContain('GOVERNED'); + expect(text).toContain('delete the row'); + }); +});