sweep(spec): three self-descriptions re-anchored to a verifiable source of truth (#6242 #4895 #4837)

.changeset/liveness-producer-and-evidence-scope.md

@@ -0,0 +1,69 @@
+---
+"@objectstack/spec": patch
+---
+
+feat(spec): a liveness `live` verdict can now cite its PRODUCER, and an entry can declare how wide its last look was (#4837, #4895)
+
+Two fields on liveness ledger entries, both optional, both validated, each
+closing a way a `live` verdict has been measurably wrong.
+
+**`producer` — a consumer is only half the call graph (#4837).** `seed.json`
+marked `Seed.env` **live**, evidence `seed-loader.ts:91`, note *"filterByEnv
+drops datasets whose env list excludes the running environment."* Every word was
+checkable against the file and the verdict was still false: line 91 really did
+call `filterByEnv(request.seeds, config.env)`, but none of the **six** call sites
+that build a `SeedLoaderRequest` passed `env` — so `config.env` was permanently
+`undefined`, the filter returned its input on its first line, and `dataset.env`
+was never read at all. The evidence pointed at the consumer; the property was
+dead at the producer. `seed-loader.test.ts` passed throughout, because the test
+supplies `config.env` itself: it exercised a mechanism nothing fed.
+
+So the criterion, now written in `liveness/README.md`: when a property's runtime
+effect depends on a **second input somebody must supply**, `live` requires
+evidence on the producer side too. `producer` carries it, and resolves through
+the same resolver as `evidence` — a repo-local path that does not exist fails
+CI, because a call-site claim nothing can falsify is exactly what the field
+exists to remove. Absence never fails (the ledger predates the field);
+`check:liveness --producer-gap` prints the worklist, and the README's table says
+which shapes actually need one — the risk class is optional config with a
+default, which always "has a value" in the type system and can still be
+`undefined` at runtime.
+
+**`evidenceScope` — how wide the last look was (#4895).** Four measured verdicts
+were reached by searching this repo alone and published as if they covered every
+consumer: `app.homePageId` ("no shell ever read it" — objectui's
+`resolveLandingRoute()` had been reading it all along), `flow.…position` (marked
+live on a designer that wrote its own `ui:{x,y}` — a false *live*, the opposite
+direction), `HttpMethod` (a scan matching only `import … from`), and
+`Notification` (objectui re-exported it with `export … from` and the real
+consumers imported from `@object-ui/types` — two hops, which no specifier match
+can see). `"evidenceScope": "in-repo" | "cross-repo"` records what was actually
+done. Absent is a worklist row; a value outside the vocabulary fails, the same
+asymmetry `verifiedAt` uses.
+
+**Entries re-verified while landing this** (not a mass re-grade — six entries
+whose call graph was closed by hand):
+
+- `seed.env` — the specimen. Evidence restamped to the live line and a `producer`
+  added: since #4704, `load()` resolves the comparison environment itself at the
+  one funnel every seeding path goes through, so call site seven cannot reopen
+  the hole.
+- `job.timeout`, `hook.retryPolicy` / `timeout` / `onError` — the same
+  "consumer reads it out of an options object built elsewhere" shape, checked
+  and **holding**: the job scheduler threads `{ retryPolicy, timeout }` into
+  `svc.schedule`, and the hook binder hands the authored hook straight to
+  `wrapDeclarativeHook`. Cited, not assumed.
+- `app.homePageId` and `book.groups[].translations` — the two surviving
+  tombstones from the #4667 retirement batch, re-verified **cross-repo** against
+  objectui `@c2fd1223` and confirmed. objectui now rejects `homePageId` in its
+  own schema with a pin test, and its book-spine interfaces declare no
+  `translations`. (The other four keys in that batch were strict-removed, so
+  they have no ledger row to date; their re-verification is recorded in the PR.)
+
+`cloud` is not reachable from an open-source checkout, so `cross-repo` means "the
+realms named in the evidence", never "everywhere" — the README says so where the
+value is defined.
+
+`producer.mts` is pure and unit-tested for the reason `orphans.mts` and
+`drill.mts` are: on the shipped ledgers these checks are almost entirely quiet,
+so a green gate proves nothing about whether they can fire.

.changeset/stack-collection-maps-answerable.md

@@ -0,0 +1,72 @@
+---
+"@objectstack/metadata": patch
+---
+
+fix(metadata,repo): every enumeration of the stack-collection set is now answerable to `stack.zod.ts`, and the artifact map stops aiming `data:` at the analytics kind (#6242)
+
+`ObjectStackDefinitionSchema` decides which collections a stack may declare — 32
+of them today. **Seven** other places re-enumerate that same set by hand (eight
+enumerations in all, because ObjectQL declares its list twice), and nothing
+compared any of them to the schema or to each other:
+
+| Enumeration | Site |
+|---|---|
+| `MAP_SUPPORTED_FIELDS` / `PLURAL_TO_SINGULAR` | `packages/spec/src/shared/metadata-collection.zod.ts` |
+| `MetadataCategoryEnum` | `packages/spec/src/kernel/package-artifact.zod.ts` |
+| `metadataArrayKeys` ×2 | `packages/objectql/src/engine.ts` |
+| `ARTIFACT_FIELD_TO_TYPE` | `packages/metadata/src/plugin.ts` |
+| `APP_CATEGORY_KEYS` | `packages/runtime/src/app-plugin.ts` |
+| `STACK_COLLECTION_COVERAGE` | `examples/app-showcase/src/coverage.ts` |
+
+They had drifted independently: `ragPipelines` mapped in three of them though no
+schema declares it; `workflows` / `approvals` / `roles` / `profiles` / `policies`
+still iterated by both ObjectQL loops after ADR-0019 / ADR-0020 / ADR-0088 /
+ADR-0090 retired them; `triggers` + `workflows` still legal artifact categories;
+19 of 32 collections absent from that enum.
+
+Every row looks like a one-line typo in isolation, and each **has** been fixed
+one line at a time before — `docs` in `ARTIFACT_FIELD_TO_TYPE`, `roles` →
+`positions` in the same map, `capabilities` in `metadataArrayKeys` — each still
+carrying its "this key was missing and it silently dropped X" comment. The cause
+is structural: `KIND_COVERAGE` is answerable to the metadata-type registry and
+fails CI when a kind is added without an entry, and the liveness ledger is
+answerable to the same registry. The collection maps were answerable to nothing.
+
+**The gate.** `pnpm check:stack-collection-maps` (root
+`scripts/check-stack-collection-maps.mjs`, wired into the lint job) derives the
+collection set from `ObjectStackDefinitionSchema` — top-level keys whose value is
+`z.array(<X>Schema)`, a mechanical rule rather than a second hand-kept list — and
+reconciles all eight enumerations against it in **both** directions. Deriving them
+is not possible today (they disagree on purpose as often as by accident: `views`
+has no `name`, `data` seeds key by `object`, `translations` is a record), so each
+deviation must instead be a waiver row **carrying its reason**, and the list is a
+ratchet: a waiver that no longer applies fails, like a stale ledger row. An
+enumeration whose symbol cannot be extracted fails too — an empty list would
+reconcile against everything.
+
+Writing it immediately found a **seventh** site the hand-audit had missed
+(`APP_CATEGORY_KEYS`) and one divergence *between* the two ObjectQL copies that
+neither list shows alone: `jobs`, `emailTemplates`, `tools` and `skills` are
+registered from a manifest and **not** from a nested plugin, so a package
+shipping them from a nested plugin registers nothing and stamps no ADR-0010
+provenance. `capabilities` was added to that copy for exactly this reason
+(#5870); nobody then asked what else the two lists disagreed about. Recorded as
+a waiver with the measurement, not fixed here — closing it changes what a nested
+plugin registers at boot.
+
+**The one code change**: `ARTIFACT_FIELD_TO_TYPE` no longer maps `data:` (the
+SEED collection) to `'dataset'` (the ADR-0021 analytics kind) — the exact name
+collision `metadata-plugin.zod.ts` warns about in prose. The entry was provably
+inert (`SeedSchema` declares no `name`, and the ingest loop skips nameless
+items), so nothing changes at runtime; what changes is that a dead pointer aimed
+at the wrong kind is gone, instead of waiting for either side to move. Not
+repointed at `'seed'`: seeds are applied by `SeedLoaderService` off the bundle,
+never registered as metadata items, so that would be new behaviour rather than a
+corrected name. The absence is now pinned by the gate.
+
+Everything else the gate reports is recorded as a waiver with its reason and left
+alone, deliberately — three of the drift rows sit on **acceptance faces**
+(`MetadataCategoryEnum` decides what a published artifact may declare) and the
+rest are `engine-core` behaviour changes owing their own verification. The value
+landing today is that all eight enumerations now have a checked relationship to
+the schema rather than an assumed one.

.github/workflows/lint.yml

@@ -133,6 +133,18 @@ jobs:
       - name: Raw control-byte guard
         run: pnpm check:nul-bytes
 
+      # Stack-collection enumerations vs the schema (#6242). `stack.zod.ts`
+      # decides which collections a stack may declare; eight other enumerations
+      # of that same set are hand-maintained (the map-format list, the
+      # plural→singular map, the artifact category enum, ObjectQL's two
+      # registration loops, the artifact-ingest field map, the runtime's
+      # app-payload probe, the showcase coverage manifest) and nothing compared
+      # any of them to the schema. Each drift instance has been fixed one line at
+      # a time before, always after a missing key silently dropped a collection.
+      # A deviation is legal here only as a waiver row carrying its reason.
+      - name: Stack-collection enumerations answerable to stack.zod.ts
+        run: pnpm check:stack-collection-maps
+
       # Docs/skills authoring guard (#2035 / ADR-0059): TS code blocks in
       # Markdown/MDX are not type-checked or ESLinted, so skills/ and
       # content/docs/ can drift back to teaching the bare `: Page = {}` literal

package.json

@@ -34,6 +34,7 @@
     "check:i18n-coverage": "node scripts/check-i18n-coverage.mjs --self-test && node scripts/check-i18n-coverage.mjs",
     "check:app-nav-i18n": "pnpm --filter @objectstack/cli run check:app-nav-i18n",
     "check:nul-bytes": "node scripts/check-nul-bytes.mjs --self-test && node scripts/check-nul-bytes.mjs",
+    "check:stack-collection-maps": "node scripts/check-stack-collection-maps.mjs --self-test && node scripts/check-stack-collection-maps.mjs",
     "check:doc-authoring": "node scripts/check-doc-authoring.mjs --self-test && node scripts/check-doc-authoring.mjs",
     "check:docs-audit-scope": "node scripts/docs-audit/affected-docs.mjs --self-test && node scripts/docs-audit/check-audit-scope.mjs --self-test && node scripts/docs-audit/check-audit-scope.mjs",
     "check:role-word": "node scripts/check-role-word.mjs",

packages/metadata/src/plugin.ts

@@ -56,7 +56,14 @@ const queryableMetadataObjects = [
 // source code (which the legacy FilesystemLoader still scans).
 const REPO_SUBDIR = '.objectstack/metadata';
 
-// Map from ObjectStackDefinition field name to MetadataType name
+// Map from ObjectStackDefinition field name to MetadataType name.
+//
+// PINNED against the schema: `scripts/check-stack-collection-maps.mjs` reconciles
+// this map with `ObjectStackDefinitionSchema` in both directions, and every
+// deviation carries a reason there (#6242). It is one of seven hand-maintained
+// enumerations of the same set, and the reason this one has a gate is its own
+// history — `docs` and `roles → positions` were each fixed here one line at a
+// time, after a missing key silently dropped a whole collection.
 const ARTIFACT_FIELD_TO_TYPE: Record<string, string> = {
     objects: 'object',
     objectExtensions: 'object_extension',
@@ -89,7 +96,15 @@ const ARTIFACT_FIELD_TO_TYPE: Record<string, string> = {
     emailTemplates: 'email_template',
     docs: 'doc',
     books: 'book',
-    data: 'dataset',
+    // `data:` (the SEED collection) is deliberately absent — #6242 row 4(a).
+    // It used to map to `'dataset'`, the ADR-0021 analytics kind: the exact name
+    // collision `metadata-plugin.zod.ts` warns about in prose. The entry never
+    // registered anything (SeedSchema declares no `name`, and the loop below
+    // skips nameless items) — a dead pointer aimed at the wrong kind, which
+    // would have begun mis-registering the day either side moved. Removed rather
+    // than repointed at `'seed'`: seeds are APPLIED by SeedLoaderService off the
+    // bundle, never registered as metadata items, so a `seed` mapping would be
+    // new behaviour rather than a corrected name.
 };
 
 // ───────────────────────────────────────────────────────────────────────────

packages/spec/liveness/README.md

@@ -143,6 +143,101 @@ re-verify rather than back-filling guesses. **For objectui-side evidence, pin
 the commit** (`objectui @732b1bf`) — `action.undoable`'s reader line numbers had
 already drifted by 28 lines one day after the issue citing them was filed.
 
+### `producer` — a consumer is only HALF the call graph (#4837)
+
+`live` means **authoring the property changes runtime behaviour**. A cited
+consumer proves something *reads* the key. It does not prove the read has an
+effect, because a read can depend on a second input that nobody supplies.
+
+`seed.json` marked `Seed.env` **live**, evidence `seed-loader.ts` line 91, note
+*"filterByEnv drops datasets whose env list excludes the running
+environment."* Every word of that was checkable against the file, and the
+verdict was still false:
+
+> Line 91 really did call `filterByEnv(request.seeds, config.env)`. But none of
+> the **six** call sites that build a `SeedLoaderRequest` — app boot, per-org
+> replay, hot reload, package apply, draft publish, marketplace install — ever
+> passed `env`. So `config.env` was permanently `undefined`, `filterByEnv`
+> returned its input on its first line, and `dataset.env` was **never read at
+> all**. The evidence pointed at the consumer; the property was dead at the
+> **producer**.
+
+The unit test made it worse rather than better: `seed-loader.test.ts` has always
+had a passing `should handle environment filtering` case — because the test
+supplies `config.env` itself. The mechanism was correct throughout; only the
+wiring was missing, and the ledger and the test both looked exclusively at the
+mechanism. This is the shape Prime Directive #10 already names in another
+context: **a `case` label is not enforcement; check the call site** (#3106).
+
+Hence the criterion:
+
+> When a property's runtime effect depends on a **second input that some
+> producer must supply**, a `live` verdict requires evidence on the producer
+> side too. Cite it in `producer`.
+
+```jsonc
+"env": {
+  "status": "live",
+  "evidence": "packages/metadata-protocol/src/seed-loader.ts:191 (filterByEnv drops …)",
+  "producer": "packages/metadata-protocol/src/seed-loader.ts:174 — load() resolves the comparison environment itself (resolveEnvConfig), rather than trusting a caller to pass it"
+}
+```
+
+`producer` resolves through **the same resolver as `evidence`** — repo-rooted
+paths must exist or CI fails; cross-repo paths are attributed and counted. A
+call-site claim nothing can falsify is precisely what this field exists to
+remove, so it does not get a weaker standard than the pointer it completes.
+
+Which entries need one. The risk is highest for **optional config with a
+default**: those always "have a value" in the type system and can still be
+`undefined` at runtime.
+
+| Shape | Needs `producer`? |
+|---|---|
+| The consumer reads the authored value directly (`hook.priority` orders hooks) | no — the author IS the producer |
+| The consumer compares the authored value against something a caller supplies (`seed.env`) | **yes** — cite who supplies it |
+| The consumer reads it out of an options/config object built elsewhere (`job.timeout`) | **yes** — cite the threading site |
+| The property is `dead` | no — there is nothing to produce |
+
+Absence never fails CI (most of the ledger predates the field, and back-filling
+guesses is the sin this records). `pnpm check:liveness --producer-gap` prints
+every `live` entry citing a consumer only — an upper bound on the debt, to be
+triaged with the table above rather than read as a defect list. A **malformed**
+value does fail.
+
+### `evidenceScope` — how wide the last look actually was (#4895)
+
+Four measured verdicts were reached by searching **this repo only**, and
+published as though they covered every consumer:
+
+| # | Verdict | What the search missed |
+|---|---|---|
+| 1 | `app.homePageId` tombstoned "no shell ever read it" | objectui's `AppContent.resolveLandingRoute()` had been reading it all along (corrected in #4709) |
+| 2 | `flow.nodes.children.position` marked live, "designer canvas layout" | the designer wrote its own `ui:{x,y}` and **nothing** read `position` — a false *live*, the opposite direction |
+| 3 | `HttpMethod` reported unused | the scan matched only `import … from` |
+| 4 | `Notification` / `NotificationConfig` removed on "zero importers" | objectui re-exported them with `export … from`, and the real consumers imported from `@object-ui/types` — **two hops**, so even a scan covering `export … from` misses it while it matches on the spec specifier |
+
+Case 4 is the one that decides the method: **no amount of text or specifier
+matching is sufficient**. A negative cross-repo claim has to follow the resolved
+symbol graph through re-export chains, or it is a guess with a citation. Every
+barrel package adds a blind spot, and the renderer repo is all barrels.
+
+`evidenceScope` records what was actually done, as data:
+
+| Value | Means |
+|---|---|
+| `in-repo` | the call graph was closed inside this repo only |
+| `cross-repo` | a named foreign realm was walked too — say **which**, in the evidence, and **pin the commit** (`objectui @c2fd1223`): `action.undoable`'s reader line numbers drifted 28 lines in one day (#3714) |
+
+Absent = scope undeclared, a worklist row rather than a failure; the field is
+younger than nearly every entry. A value outside the vocabulary FAILS, the same
+asymmetry as `verifiedAt` — a value the parser cannot read would silently exempt
+that entry from every future sweep.
+
+⚠️ Neither `cross-repo` value in the tree today covers **`cloud`**: the closed
+runtime is not reachable from an open-source checkout, so a `cross-repo` claim
+means "the realms named in the evidence", never "everywhere".
+
 ### ⚠️ An authoring/preview renderer is NOT a runtime consumer
 
 `live` means **authoring the property changes runtime behaviour**. A Studio
@@ -522,10 +617,15 @@ over-share.
   silently. Same "pure + unit-tested" reasoning as `orphans.mts`, for the same reason.
 - `../scripts/liveness/check-empty-state.mts` — the empty-state gate (above);
   `empty-state-registry.mts` is its source of truth.
+- `../scripts/liveness/producer.mts` — the `producer` / `evidenceScope` fold
+  (#4837 / #4895). Pure + unit-tested for the same reason as `orphans.mts`: on
+  the shipped ledgers these checks are almost entirely quiet, so a green gate
+  proves nothing about whether they can fire.
 
 ```bash
 pnpm --filter @objectstack/spec check:liveness               # run the gate
 tsx packages/spec/scripts/liveness/check-liveness.mts --dump field   # inventory a type (seeding aid)
+tsx packages/spec/scripts/liveness/check-liveness.mts --producer-gap # live entries citing a consumer only
 ```
 
 CI: `.github/workflows/spec-liveness-check.yml` runs on PRs touching `packages/spec/**`.