From fe3990f6e28113b9e1d9ce5b599ce674f7d07569 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 18:15:43 +0000 Subject: [PATCH 1/4] feat(spec)!: close ApiEndpointSchema against undeclared keys, and name the author state on the upgrade path (#5384, #5227) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `api` became a registered metadata type at #5312, making `endpoint.zod.ts` an authoring surface (`defineStack({ apis })`, the Studio form, `PUT /meta/api/:name`) while it was still an open `z.object`. A `cacheTTL` / `objectParam` / `outputMappings` typo parsed green, published green, and served without the policy or projection its author wrote. The shape is `strictObject` now, with curated wrong-layer prescriptions for `namespace` (ADR-0121 D2 — derived from `manifest.namespace`, never per-endpoint) and the six stored-envelope bookkeeping keys. The ORDER is the record: closing this was measured and refused first, because the same schema parsed STORED rows carrying `packageId` / `state`. #5309 (PR #6576) paid that debt at the layer that owned it (`peelStoredEnvelope`), after which the conversion was an ordinary #4001 one. `ApiEndpointSchema` never learned a bookkeeping key. Corpus measured before landing: examples/* = 2 endpoints, cloud = 0 endpoints, zero undeclared keys either side, so direction A carried and no key had to stay. `api` leaves STILL_STRIP (closed 24 -> 25 of 26; `view` alone remains) and the CLI gate row moves NOT_YET_CLOSED -> GATED_AT. #5227 needed no schema change: ADR-0122 phase 2 (PR #6279) already made `ApiEndpoint` the author state, so omitting `authRequired` compiles. The `declarative-apis-endpoints-live` upgrade-guide entry now says so with a usable annotation. An `ApiEndpointInput` alias is deliberately NOT added — after the flip it would be a synonym, which ADR-0122 D3 forbids and `check:spec-parsed-alias` rule 4 rejects. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011sGk4SKHqGRgmmqUok1P8M --- .changeset/api-endpoint-schema-strict.md | 50 ++++++ .../2026-07-unknown-key-strictness-ledger.md | 2 +- docs/protocol-upgrade-guide.md | 2 +- .../test/metadata-type-schema-gate.test.ts | 59 +++++-- .../src/publish-endpoint-gate.test.ts | 46 +++++- packages/metadata/src/stored-envelope.test.ts | 50 ++++++ packages/spec/spec-changes.json | 4 +- packages/spec/src/api/endpoint.test.ts | 144 ++++++++++++++++++ packages/spec/src/api/endpoint.zod.ts | 139 ++++++++++++++--- .../src/kernel/metadata-type-schemas.test.ts | 65 +++++--- packages/spec/src/migrations/registry.ts | 10 +- 11 files changed, 502 insertions(+), 69 deletions(-) create mode 100644 .changeset/api-endpoint-schema-strict.md diff --git a/.changeset/api-endpoint-schema-strict.md b/.changeset/api-endpoint-schema-strict.md new file mode 100644 index 0000000000..f703a605d3 --- /dev/null +++ b/.changeset/api-endpoint-schema-strict.md @@ -0,0 +1,50 @@ +--- +"@objectstack/spec": minor +--- + +`ApiEndpointSchema` rejects undeclared keys (#5384), and the author-state type is named on the upgrade path (#5227) + +`api` became a registered metadata type at #5312, which made +`packages/spec/src/api/endpoint.zod.ts` an AUTHORING surface — `defineStack({ apis })`, +the Studio metadata-admin form, and `PUT /meta/api/:name`'s 422 — while it was still a +plain open `z.object`. An undeclared key was therefore dropped on every path: a +`cacheTTL` / `objectParam` / `outputMappings` typo parsed green, published green, and the +endpoint then served without the policy or projection its author wrote. That direction is +fail-safe for `authRequired` alone (an unrecognized spelling leaves the default `true` +standing); it was never fail-safe for the mapping, cache and rate-limit blocks. + +The shape is now `strictObject`, so an undeclared key is a named rejection carrying the +surface, the offending key and a rename. Two curated wrong-layer prescriptions ship with +it: + +- **`namespace`** — ADR-0121 D2 derives the namespace segment of `path` from + `manifest.namespace`; it has never been per-endpoint, so the rejection points at the + manifest instead of suggesting a rename. +- **the six stored-envelope bookkeeping keys** (`packageId`, `state`, `version`, + `published*`) — written onto the stored ROW by `register` / `publishPackage`, never onto + a declaration. + +**The order this landed in is the part worth keeping.** Closing the shape was measured and +REFUSED first: the same schema parsed STORED rows at `buildEndpointIndex` and +`gateApiItemsForPublish`, so a naked `strictObject` failed every row with +`unrecognized_keys: ['packageId', 'state']` — the load-time backstop excluded the endpoint +(404) and the publish gate reported a schema error in place of the ADR-0121 D6 verdict it +exists to give. The debt was real and it was not in this vocabulary, so #5309 (PR #6576) +paid it at the layer that owned it (`peelStoredEnvelope`). `ApiEndpointSchema` never +learned a bookkeeping key. + +**Breaking for metadata that was already silently broken.** An `apis:` entry carrying an +undeclared key now fails `objectstack validate`, `objectstack build` and the metadata write +path instead of publishing with the key discarded. Measured before landing: the example +corpus (2 endpoints) and the `cloud` repository (0 endpoints) carry zero undeclared keys, +so nothing in-tree changes verdict. + +`api` also leaves the #4001 campaign's `STILL_STRIP` list — closed registered types 24 → 25 +of 26, with `view` the only entry left — and the CLI metadata gate's row moves from +`NOT_YET_CLOSED` into `GATED_AT`. + +**#5227** — no schema change. `ApiEndpoint` already denotes the AUTHOR state after ADR-0122 +phase 2, so omitting `authRequired` compiles; what was missing was anywhere saying so. The +`declarative-apis-endpoints-live` upgrade-guide entry, whose whole safety argument is that +"an omission is SAFE", now carries the type annotation that makes the omission writable: +annotate declarations `ApiEndpoint`, hold parse results as `ApiEndpointParsed`. diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.md b/docs/audits/2026-07-unknown-key-strictness-ledger.md index 74758fbb27..f9f780a266 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.md @@ -1261,7 +1261,7 @@ rest at #5107. | Dir | Dominant class | Rationale | |---|---|---| -| `api/` | wire | REST/GraphQL request/response contracts — tolerant by design | +| `api/` | **mixed · `endpoint.zod.ts` authorable, the rest wire** | ⚠️ **Split at #5384 — and the flat `wire` verdict this row used to carry was correct when written and then silently expired, which is the finding.** The rest of the directory is unchanged: REST/GraphQL request/response contracts, tolerant by design. But `endpoint.zod.ts` stopped being one of them at **#5312**, which registered `api` as a metadata type (`DEFAULT_METADATA_TYPE_REGISTRY` / `BUILTIN_METADATA_TYPE_SCHEMAS`) — from that moment `ApiEndpointSchema` was simultaneously an AUTHORING surface (`defineStack({ apis })`, the Studio metadata-admin form, `PUT /meta/api/:name`'s 422) and a wire shape, while this row still told every reader the whole directory was "tolerant by design". A row is read as licence, which is exactly what this ledger's own gate exists to prevent. `ApiEndpointSchema` is now `strictObject` (#5384): an undeclared key on an endpoint is a named rejection carrying the surface, the offending key and a rename, instead of a silent strip that let a `cacheTTL` / `objectParam` / `outputMappings` typo publish green and serve without the policy or projection its author wrote. Two curated wrong-layer pointers ship with it: **`namespace`** (ADR-0121 D2 — the namespace segment of `path` is derived from `manifest.namespace` and has never been per-endpoint; `publish-endpoint-gate.test.ts` pins that the gate does not believe it) and the six **stored-envelope bookkeeping** keys (`packageId`, `state`, `version`, `published*`). ⚠️ **The order is the part worth keeping.** Closing this shape was measured and REFUSED first (2026-08-05, maintainer): the same schema parsed STORED rows at `buildEndpointIndex` and `gateApiItemsForPublish`, so a naked `strictObject` failed every row with `unrecognized_keys: ['packageId', 'state']` — the load-time backstop excluded the endpoint (404) and the publish gate reported a schema error in place of the ADR-0121 D6 verdict it exists to give, 11 tests red in `packages/metadata`. The debt was real and it was NOT in this vocabulary, so it was paid at the layer that owned it: **#5309 (PR #6576)** peeled the envelope off before the body parse (`peelStoredEnvelope`), after which a strict probe left exactly ONE red — a fixture planting an authored `namespace`, re-spelled by #5384 rather than deleted. Teaching `ApiEndpointSchema` two bookkeeping keys to buy strictness would have made the authoring contract describe the storage layer; that trade was refused and did not have to be made. `api` left `STILL_STRIP` (`kernel/metadata-type-schemas.test.ts`) with this change — closed registered types 24 → 25 of 26 — and the CLI gate row moved from `NOT_YET_CLOSED` into `GATED_AT` (`packages/cli/test/metadata-type-schema-gate.test.ts`), the deliberate ratchet step that file's own note asked for. `view` is now the only entry left on `STILL_STRIP` | | `system/` | mixed | manifest/datasource blocks are authored; runtime envelopes are wire | | `kernel/` | wire | plugin/kernel contracts, code-to-code | | `cloud/` | wire | multi-tenant runtime | diff --git a/docs/protocol-upgrade-guide.md b/docs/protocol-upgrade-guide.md index 5c996e65b8..a693ba508d 100644 --- a/docs/protocol-upgrade-guide.md +++ b/docs/protocol-upgrade-guide.md @@ -345,7 +345,7 @@ One entry in this step is not a removal at all but a SECURE-DEFAULT FLIP, the sh - Why not automatic: The five schemas declared the "Dynamic Loading" capability — runtime load / unload / reload of plugins without a kernel restart, with sandboxing, integrity hashes, drain strategies and dependent-cascade policy — and NOTHING implemented it. A bare-name scan of objectstack, cloud and objectui found zero references outside this package's own declaration, its unit tests and the generated artifacts: no runtime ever received a `DynamicLoadRequest`, performed a load/unload, or produced a `DynamicPluginResult`. That is the ADR-0049 false-compliance shape at its most inviting to an AI author (ADR-0033), who reads `DynamicLoadRequestSchema` in the published IDE bundle as proof the platform hot-loads plugins and constructs a request that parses clean and is received by nobody (#3950: an exported schema with no consumer is read as a capability). The #3896 follow-up removed this module's discovery/sandbox config island and left these five in place explicitly — "operation contracts, not security promises; the enforce-or-remove call on them is a design decision rather than a correction" — but that suspension lived only in a changeset paragraph with no issue carrying it. #4834 is that decision, answered REMOVE. `experimental` was considered and rejected: it is only `.describe()` prose and cannot stop an import, the weakest of the three ADR-0049 channels. None of the five is stored metadata — they are root request/result payload shapes embedded in no parent schema and parsed against no metadata document — so no `sys_metadata` row can carry one and there is no source for the D2 chain to rewrite; this entry is the D3 record. The removal also subsumes the kernel half of `plugin-activation-events-retired` (#4657): that tombstone goes with the shape that carried it. ADR-0049, #4834. - Done when: No code imports `DynamicLoadRequestSchema`, `DynamicUnloadRequestSchema`, `DynamicPluginResultSchema`, `PluginSourceSchema`, `DynamicPluginOperationSchema` or any of their type aliases (`DynamicLoadRequest`, `DynamicUnloadRequest`, `DynamicPluginResult`, `PluginSource`, `DynamicPluginOperation`, `DynamicLoadRequestInput`, `DynamicUnloadRequestInput`) from `@objectstack/spec` or `@objectstack/spec/kernel` — every one is TS2305 after upgrade, on every public entry (pinned by symbol identity in `plugin-runtime-retirement.test.ts`). Nothing regresses at runtime, because nothing called anything: a caller that believed it was hot-loading a plugin was already only building an object. Boot-time composition through `defineStack` is unchanged. - **`declarative-apis-endpoints-live`** — `stack.apis[] (every declared ApiEndpoint — REVIEW REQUIRED BEFORE UPGRADING)` → the same declarations, re-read as LIVE HTTP routes: `path` moved under `/api/v1/apps//`, and every entry that declares `authRequired: false` re-confirmed as an intentionally anonymous endpoint carrying `rateLimit: { enabled: true, … }` - - Why not automatic: This is the one protocol-17 entry that turns metadata ON rather than off, so read it as a SECURITY review item and not as a rename. Before 17 the declarative endpoint surface executed NOTHING: no route was mounted for a declared `path`, no matcher existed, and every key — `authRequired` included — parsed green and gated nothing (#4936, which refused a non-empty `apis:` outright for exactly that reason). Protocol 17 ships the executor (#5040) and narrows that refusal to a per-endpoint publish gate: an endpoint that PASSES the gate is mounted and serves real traffic as soon as the stack is published. So an `apis:` block written against an older major — or one restored from a pre-#4936 source, or authored from a doc that predates the refusal — changes meaning without changing a byte: what used to be inert documentation becomes an execution entry point into the data and automation pipelines. Nothing about that transition can be applied mechanically, because the judgment it needs is "did the author of this endpoint mean for the internet to reach it?" — and the one key where a wrong answer is unrecoverable is `authRequired`. Its schema default is `true`, so an omission is SAFE and needs no review; an EXPLICIT `authRequired: false` is the only thing that opens anonymous access, and under ADR-0121 D6 it now also requires an armed `rateLimit` (`enabled: true` — the key defaults to `false`, so a budget written without it meters nothing) or the stack refuses to publish. Grep every `apis:` entry for `authRequired: false` before you upgrade, delete the ones that were never meant to be public, and arm a budget on the ones that were. The path move is the mechanical-looking half and is still yours: ADR-0121 D1/D2 confine a declared path to your own namespace carve-out (`/api/v1/apps//…`), the namespace comes from an explicit `manifest.namespace` with no derivation fallback, and the subpath is the only part you name — rewriting it for you would silently change a URL third parties call. + - Why not automatic: This is the one protocol-17 entry that turns metadata ON rather than off, so read it as a SECURITY review item and not as a rename. Before 17 the declarative endpoint surface executed NOTHING: no route was mounted for a declared `path`, no matcher existed, and every key — `authRequired` included — parsed green and gated nothing (#4936, which refused a non-empty `apis:` outright for exactly that reason). Protocol 17 ships the executor (#5040) and narrows that refusal to a per-endpoint publish gate: an endpoint that PASSES the gate is mounted and serves real traffic as soon as the stack is published. So an `apis:` block written against an older major — or one restored from a pre-#4936 source, or authored from a doc that predates the refusal — changes meaning without changing a byte: what used to be inert documentation becomes an execution entry point into the data and automation pipelines. Nothing about that transition can be applied mechanically, because the judgment it needs is "did the author of this endpoint mean for the internet to reach it?" — and the one key where a wrong answer is unrecoverable is `authRequired`. Its schema default is `true`, so an omission is SAFE and needs no review; an EXPLICIT `authRequired: false` is the only thing that opens anonymous access, and under ADR-0121 D6 it now also requires an armed `rateLimit` (`enabled: true` — the key defaults to `false`, so a budget written without it meters nothing) or the stack refuses to publish. ⚠️ If you author endpoints in TypeScript, annotate them with `ApiEndpoint` — the AUTHOR state — so that omitting `authRequired` compiles: `const e: ApiEndpoint = { name, path, method, type, target }` is legal and is the safe shape this paragraph prescribes. `ApiEndpointParsed` is the POST-parse type (defaults materialized, ADR-0122), where `authRequired` is required — annotating a declaration with it forces you to write the key out, and being made to think about a key whose only unrecoverable value is `false` is the one thing this entry is trying to avoid (#5227). Hold a parse RESULT with `ApiEndpointParsed`; write declarations as `ApiEndpoint`. Grep every `apis:` entry for `authRequired: false` before you upgrade, delete the ones that were never meant to be public, and arm a budget on the ones that were. The path move is the mechanical-looking half and is still yours: ADR-0121 D1/D2 confine a declared path to your own namespace carve-out (`/api/v1/apps//…`), the namespace comes from an explicit `manifest.namespace` with no derivation fallback, and the subpath is the only part you name — rewriting it for you would silently change a URL third parties call. - Done when: You have READ every entry of every `apis:` block, not just the ones that fail to publish. Concretely: (1) each declared `path` is `/api/v1/apps//` and the stack declares that `manifest.namespace` explicitly; (2) every entry declaring `authRequired: false` is one you INTEND to be reachable without a session, and each carries `rateLimit: { enabled: true, windowMs, maxRequests }` — entries that were not intended to be anonymous have the key removed so the safe default (`true`) applies; (3) `objectstack validate` passes, which also proves no endpoint declares a shape 17.x cannot execute (`type: script` / `proxy`, mapping `transform`, an `object_operation` missing `objectParams`, `cacheTtl` on a non-GET method, `inputMapping` on find/get/delete, or two endpoints claiming one METHOD + path); and (4) after publishing, each endpoint answers as you expect — an anonymous request to a session-only endpoint returns 401 rather than data. - **`ui-widget-i18n-family-retired`** — `ui.widgetManifest / ui.widgetLifecycle / ui.widgetEvent / ui.widgetProperty / ui.widgetSource / ui.i18nObject / ui.pluralRule / ui.numberFormat / ui.dateFormat / ui.localeConfig (the widget-registration vocabulary of ui/widget.zod.ts, and the five doorless shapes of ui/i18n.zod.ts — 10 defs, 26 exported names)` → (removed — there is no replacement key, because there was never a key. A custom field widget is still named the same way it always was: `field.widget` is a plain string naming a component the RENDERER has registered, and objectui's registry has always carried its own runtime manifest for that (`RuntimeWidgetManifest` / `RuntimeWidgetSource` in `@object-ui/types`, objectui#3161 / #4115), which models different keys and never derived from these. For localisation: write the default-language string on `label` / `description` — the framework generates the translation key at registration time from the naming convention — and put translations in translation files, which is the LIVE `system/translation.zod.ts` surface. Widget registration and locale formatting as authorable protocol metadata return via the ENFORCE route of ADR-0049 through a new ADR — the registry / loader / formatter first, the vocabulary second) - Why not automatic: `ui/widget.zod.ts` published a complete widget-registration vocabulary — a manifest with lifecycle hooks, custom events, configurable properties and an npm/remote/inline implementation-source union — and `ui/i18n.zod.ts` published a structured-label, plural-rule and locale-formatting vocabulary. NOTHING in the protocol carried either. Three independent measurements, re-run on `origin/main` immediately before the removal with their controls passing in the SAME run: (1) no module under `packages/spec/src` imported `widget.zod` at all, and the only imports of `i18n.zod` anywhere name `I18nLabelSchema` / `AriaPropsSchema` (both KEPT), so no schema declared a carrier key — `field.widget` is a `z.string()` naming a registered component and has never referenced `WidgetManifest`; (2) a BFS over the in-memory Zod graph from all 24 metadata-type roots plus `defineStack`'s `ObjectStackSchema` reached none of them, while `PageSchema` / `ObjectListViewSchema` resolved `direct` in the same run and a synthetic carrier flipped every one of them; (3) zero `.parse()` / `.safeParse()` in objectstack, objectui or cloud outside these files' own unit tests. `NumberFormat` / `DateFormat` DID have a carrier key (`LocaleConfig.numberFormat` / `.dateFormat`) but the carrier was itself doorless, so the subtree was `no door` rather than `no gate` and goes whole — leaving the two leaves behind would strand exported schemas with no consumer (#3950). `I18nObjectSchema` was additionally superseded by its own file-neighbour: `I18nLabelSchema`'s documentation already says translation keys are generated at registration time and translations live in translation files, and the live translation surface is `system/translation.zod.ts`, which uses none of these shapes. The 2026-08-06 ruling weighed giving them a carrier (option B) and rejected it: that is a feature with a registry and a renderer behind it, not ledger clean-up. Tightening them to `strictObject` was rejected earlier and explicitly (#4001 批 16) — strictness is a property of a PARSE and there is no parse, so it would spend a breaking change to leave "a precisely validated dead slot, the more convincing lie" (#4583). With no carrier key there is nothing to tombstone and no `sys_metadata` row or source file for a D2 conversion to rewrite: this entry is the D3 record, route 3, the same shape as #4988 (the ui/ interaction config family), #4834 (kernel plugin-runtime family) and #4938 (`HttpServerConfig`). ⚠️ `WidgetManifest.performance`'s own `retiredKey()` tombstone (#3896 close-out) is SUBSUMED here, the #4657/#4834 way: it goes with the shape that carried it, which is strictly stronger than the tombstone, because there is no longer a manifest to author the key INTO. ⚠️ One of the nine widget sites is deliberately NOT retired. `FieldWidgetPropsSchema` survives: it is a REACT PROPS CONTRACT rather than authorable metadata (it never appeared in `authorable-surface/` or `json-schema.manifest/` — its `onChange` is a `z.function()`), so "zero parse" is its design and not its defect, and it acquired a live cross-repo compile-time consumer one day before 批 16 measured: objectui PR #3289 (2026-08-03) renamed `@object-ui/fields`' validation slot onto the spec's `error` with no alias, the form renderer began producing it, and `packages/fields/src/__tests__/spec-symbol-batch7.test.ts` pins the shape against `import type { FieldWidgetProps } from '@objectstack/spec/ui'` as an intentional tripwire. Re-verified on objectui `origin/main` 2026-08-07. ADR-0049, #5055. diff --git a/packages/cli/test/metadata-type-schema-gate.test.ts b/packages/cli/test/metadata-type-schema-gate.test.ts index 091c416817..a2428f8421 100644 --- a/packages/cli/test/metadata-type-schema-gate.test.ts +++ b/packages/cli/test/metadata-type-schema-gate.test.ts @@ -24,9 +24,10 @@ * A. the CLI parses through the registry's schemas — one undeclared key, the * same verdict from both gates, for every registered metadata type. Three * carriers are structurally different and are asserted at their real - * positions; one type (`api`) has a schema #4001 has not closed yet, so - * there the claim is AGREEMENT plus "the author is still told", with the - * gap filed rather than papered over (#5384); + * positions. The one type that was NOT closed when this file was written + * (`api`) closed at **#5384**, so its weaker claim — AGREEMENT plus "the + * author is still told" — has been replaced by the full one, both gates + * REJECT, and the not-yet-closed table is now empty; * B. the commands GATE on that parse — the issue's undeclared-key repro and * the #4001 batch-13 `responsiveStyles.large` → `.lg` negative control, * run through the real binary: non-zero exit, prescription in the output, @@ -91,6 +92,15 @@ const GATED_AT: Readonly> = { agent: 'agents', tool: 'tools', skill: 'skills', + // [#5384] `api` GRADUATED here from NOT_YET_CLOSED — the deliberate ratchet + // step this file's own note asked for. `ApiEndpointSchema` is `strictObject` + // as of #5384, so the stack's `apis:` array and the registry schema now reject + // an undeclared key identically, and "both gates agree, and the author is + // still told" is no longer the strongest claim available: "both gates REJECT" + // is. The blocker was never this vocabulary — #5309 (PR #6576) moved the + // stored envelope (`packageId` / `state` / …) off the body first, which is + // what made the closure a vocabulary change instead of a storage compromise. + api: 'apis', }; /** @@ -113,22 +123,28 @@ const STRUCTURAL_EXCEPTIONS: Readonly> = { * Registered types whose SCHEMA is not closed yet, so "both gates reject" is * not the claim to make — "both gates agree, and the author is still told" is. * - * `api` is the live one. #5312 registered the type and the stack authors it at + * **EMPTY as of #5384, and that is a hard zero rather than a gap.** `api` was + * the one live row: #5312 registered the type and the stack authors it at * `apis:` (ADR-0121; note the neighbouring singular `api:` block, which is - * server-facing REST config, not metadata), but `ApiEndpointSchema` is still a - * plain `z.object` — the #4001 campaign has not reached it, and the strictness - * ledger still files all of `api/` as wire. An undeclared key on an endpoint - * is therefore DROPPED on both the write path and here, identically. Filed as - * #5384 (sub-issue of #4001) rather than papered over. + * server-facing REST config, not metadata), while `ApiEndpointSchema` was still + * a plain `z.object` — an undeclared key on an endpoint was DROPPED on both the + * write path and here, identically. What this table asserted for it was the + * weaker pair: the CLI is no looser than the write path, and the #3786 + * pre-parse layer still names the key, so the author is not left with silence. * - * What is asserted instead: the CLI is no looser than the write path, and the - * #3786 pre-parse layer still names the key, so the author is not left with - * silence. When #5384 closes the shape, the agreement assertions below go red - * — that is the ratchet working; move the row into `GATED_AT` then. + * That row is GONE, in the direction the note above it predicted: #5384 closed + * the shape with `strictObject`, the agreement assertions went red, and `api` + * moved into `GATED_AT` — the deliberate ratchet step, not a surprise. The + * blocker was never the endpoint vocabulary; #5309 (PR #6576) split the stored + * envelope off the authored body first, and only then was closing it a + * vocabulary change. + * + * The table and its assertions STAY, for the next type that registers ahead of + * its closure — which is how `api` arrived in the first place. While it is + * empty the case below asserts the emptiness explicitly, because a loop over + * nothing is a green that proves nothing. */ -const NOT_YET_CLOSED: Readonly> = { - api: { collection: 'apis', tracking: '#5384 (sub-issue of #4001)' }, -}; +const NOT_YET_CLOSED: Readonly> = {}; /** Every `unrecognized_keys` issue naming `INJECTED_KEY`, with its path. */ function undeclaredKeyRejections(result: { success: boolean; error?: any }): string[] { @@ -295,6 +311,17 @@ describe('the CLI parses metadata through the registry schemas (#5000)', () => { }); it('is no looser than the write path on a type #4001 has not closed, and still names the key', () => { + // [#5384] The table is EMPTY, so the loop below runs zero times — and a + // green from a loop over nothing proves nothing. Assert the zero itself, so + // the state is pinned rather than assumed: if a type is ever added back + // here, this line is the one that must be consciously edited, and the loop + // resumes covering it. + expect( + Object.keys(NOT_YET_CLOSED), + 'a registered type is back on the not-yet-closed list — the loop below now covers it, and ' + + 'this expectation records the regression deliberately rather than passing vacuously', + ).toEqual([]); + for (const [type, { collection, tracking }] of Object.entries(NOT_YET_CLOSED)) { const registry = getMetadataTypeSchema(type); expect(registry, `no registered schema for '${type}'`).toBeDefined(); diff --git a/packages/metadata/src/publish-endpoint-gate.test.ts b/packages/metadata/src/publish-endpoint-gate.test.ts index 65e532d306..67f403c891 100644 --- a/packages/metadata/src/publish-endpoint-gate.test.ts +++ b/packages/metadata/src/publish-endpoint-gate.test.ts @@ -194,14 +194,58 @@ describe('#5189 — publishPackage gates `api` items', () => { expect(precondition!.message).toContain('publishPackage(id, { namespace })'); }); - it('does NOT infer the namespace from the item being judged', async () => { + it('does NOT infer the namespace from the item being judged — the key cannot even be authored (#5384)', async () => { // The item declares a `namespace` field and a matching path; believing it // would make the carve-out gate vacuous. + // + // ⚠️ [#5384] The MECHANISM that guarantees this changed, and the change is + // a strengthening — so the assertion moved with it rather than being + // deleted. Before: `ApiEndpointSchema` was an open `z.object`, the key + // parsed through and was silently stripped, and the gate went on reading + // the manifest — so the proof was "the gate ignores it". Now the schema is + // `strictObject` and `namespace` is refused BY NAME before the gate runs: + // the gate cannot infer a namespace from the item because the item can no + // longer express one. `namespace` was deliberately NOT peeled as a stored + // envelope key in #5309 — it is authored here, and peeling it would hide + // exactly the typo class #5384 exists to catch. + // + // Which refusal SPEAKS is therefore what changed; that publish refuses is + // unchanged, and both halves are asserted. await manager.register('api', 'list_things', apiItem({ namespace: NS })); + const result = await manager.publishPackage(PKG); + expect(result.success).toBe(false); + + // The named verdict entry — `publishPackage` speaks in `validationErrors[]`, + // not in an ADR-0112 envelope, so the entry is what carries the refusal. + const refusal = result.validationErrors!.find(e => e.message.includes('ApiEndpointSchema')); + expect( + refusal, + 'the closed shape must refuse the authored `namespace` key by name', + ).toBeDefined(); + expect(refusal!.name).toBe('list_things'); + expect(refusal!.type).toBe('api'); + // The offending key is echoed back … + expect(refusal!.message).toContain('namespace'); + // … with the ADR-0121 D2 wrong-layer prescription, not a bare "unrecognized". + expect(refusal!.message).toContain('manifest.namespace'); + }); + + it('the carve-out gate is not vacuous: a clean item with no namespace supplied still hits the precondition', async () => { + // CONTROL for the case above. Without it, that test would keep passing if + // the shape started refusing EVERY item for some unrelated reason — the + // failure mode this file's own §1 warns about. Same call, same absent + // namespace, but a body the schema accepts: the refusal that speaks must + // be the manifest precondition, and it must NOT be a schema error. + await manager.register('api', 'list_things', apiItem()); + const result = await manager.publishPackage(PKG); expect(result.success).toBe(false); expect(result.validationErrors!.some(e => e.message.includes('manifest.namespace'))).toBe(true); + expect( + result.validationErrors!.some(e => e.message.includes('ApiEndpointSchema')), + 'a clean endpoint body must not produce a schema error', + ).toBe(false); }); }); diff --git a/packages/metadata/src/stored-envelope.test.ts b/packages/metadata/src/stored-envelope.test.ts index d767803cf2..b1eb840b41 100644 --- a/packages/metadata/src/stored-envelope.test.ts +++ b/packages/metadata/src/stored-envelope.test.ts @@ -16,6 +16,7 @@ */ import { describe, it, expect } from 'vitest'; +import { ApiEndpointSchema } from '@objectstack/spec/api'; import { peelStoredEnvelope, storedItemName, @@ -163,3 +164,52 @@ describe('storedItemName — the row’s own name, for a diagnostic', () => { expect(storedItemName(peelStoredEnvelope(undefined))).toBeUndefined(); }); }); + +/** + * [#5384] The other half of the split, asserted from this side. + * + * `ApiEndpointSchema` closed with `strictObject`, and its `guidance` table + * carries a wrong-layer prescription for each stored-envelope key: meeting one + * on an AUTHORED declaration means somebody hand-wrote bookkeeping into a + * document, where it has never configured anything. That table is a second copy + * of `STORED_ENVELOPE_KEYS`, one package away, and `packages/spec` may not + * import from here (Prime Directive #2) — so the copies are pinned equal + * BEHAVIOURALLY, from the side that is allowed to see both. + * + * The drift this catches is one-directional and silent: a seventh bookkeeping + * key added to `STORED_ENVELOPE_KEYS` would be peeled correctly on the stored + * paths (so every test above stays green) while an author who wrote it by hand + * got the generic "unrecognized key" instead of the prescription — the rejection + * arriving without the upgrade, which is finding 7's shape. + */ +describe('[#5384] every stored-envelope key carries a wrong-layer prescription on the authoring surface', () => { + it.each([...STORED_ENVELOPE_KEYS])('`%s` is refused by name, with the bookkeeping prescription', (key) => { + const result = ApiEndpointSchema.safeParse({ ...AUTHORED, [key]: 'anything' }); + + expect(result.success, `\`${key}\` parsed clean on an authored endpoint`).toBe(false); + const unknown = result.error!.issues.find( + (i) => i.code === 'unrecognized_keys' && (i as { keys?: string[] }).keys?.includes(key), + ); + expect(unknown, `\`${key}\` was not reported as an unrecognized key`).toBeDefined(); + // The prescription, not merely a rejection: it must name the layer that + // owns the key and tell the author what to do instead. + expect(unknown!.message).toContain('storage bookkeeping'); + expect(unknown!.message).toContain('#5309'); + }); + + it('CONTROL — the same body without a bookkeeping key parses, so the case above cannot pass vacuously', () => { + const clean = ApiEndpointSchema.safeParse({ ...AUTHORED }); + expect(clean.success).toBe(true); + }); + + it('CONTROL — `metadata` (the BODY key) is not bookkeeping and gets no prescription', () => { + // STORED_BODY_KEY is the envelope's carrier, never a declared endpoint key. + // It must still be refused — but as a plain unknown key, because telling an + // author it is "storage bookkeeping to remove" would misdescribe it. + const result = ApiEndpointSchema.safeParse({ ...AUTHORED, [STORED_BODY_KEY]: {} }); + expect(result.success).toBe(false); + const unknown = result.error!.issues.find((i) => i.code === 'unrecognized_keys'); + expect(unknown).toBeDefined(); + expect(unknown!.message).not.toContain('storage bookkeeping'); + }); +}); diff --git a/packages/spec/spec-changes.json b/packages/spec/spec-changes.json index b3a2ca4d7a..eb10f7902e 100644 --- a/packages/spec/spec-changes.json +++ b/packages/spec/spec-changes.json @@ -593,7 +593,7 @@ "replacement": "the same declarations, re-read as LIVE HTTP routes: `path` moved under `/api/v1/apps//`, and every entry that declares `authRequired: false` re-confirmed as an intentionally anonymous endpoint carrying `rateLimit: { enabled: true, … }`", "migrationId": "declarative-apis-endpoints-live", "toMajor": 17, - "rationale": "This is the one protocol-17 entry that turns metadata ON rather than off, so read it as a SECURITY review item and not as a rename. Before 17 the declarative endpoint surface executed NOTHING: no route was mounted for a declared `path`, no matcher existed, and every key — `authRequired` included — parsed green and gated nothing (#4936, which refused a non-empty `apis:` outright for exactly that reason). Protocol 17 ships the executor (#5040) and narrows that refusal to a per-endpoint publish gate: an endpoint that PASSES the gate is mounted and serves real traffic as soon as the stack is published. So an `apis:` block written against an older major — or one restored from a pre-#4936 source, or authored from a doc that predates the refusal — changes meaning without changing a byte: what used to be inert documentation becomes an execution entry point into the data and automation pipelines. Nothing about that transition can be applied mechanically, because the judgment it needs is \"did the author of this endpoint mean for the internet to reach it?\" — and the one key where a wrong answer is unrecoverable is `authRequired`. Its schema default is `true`, so an omission is SAFE and needs no review; an EXPLICIT `authRequired: false` is the only thing that opens anonymous access, and under ADR-0121 D6 it now also requires an armed `rateLimit` (`enabled: true` — the key defaults to `false`, so a budget written without it meters nothing) or the stack refuses to publish. Grep every `apis:` entry for `authRequired: false` before you upgrade, delete the ones that were never meant to be public, and arm a budget on the ones that were. The path move is the mechanical-looking half and is still yours: ADR-0121 D1/D2 confine a declared path to your own namespace carve-out (`/api/v1/apps//…`), the namespace comes from an explicit `manifest.namespace` with no derivation fallback, and the subpath is the only part you name — rewriting it for you would silently change a URL third parties call." + "rationale": "This is the one protocol-17 entry that turns metadata ON rather than off, so read it as a SECURITY review item and not as a rename. Before 17 the declarative endpoint surface executed NOTHING: no route was mounted for a declared `path`, no matcher existed, and every key — `authRequired` included — parsed green and gated nothing (#4936, which refused a non-empty `apis:` outright for exactly that reason). Protocol 17 ships the executor (#5040) and narrows that refusal to a per-endpoint publish gate: an endpoint that PASSES the gate is mounted and serves real traffic as soon as the stack is published. So an `apis:` block written against an older major — or one restored from a pre-#4936 source, or authored from a doc that predates the refusal — changes meaning without changing a byte: what used to be inert documentation becomes an execution entry point into the data and automation pipelines. Nothing about that transition can be applied mechanically, because the judgment it needs is \"did the author of this endpoint mean for the internet to reach it?\" — and the one key where a wrong answer is unrecoverable is `authRequired`. Its schema default is `true`, so an omission is SAFE and needs no review; an EXPLICIT `authRequired: false` is the only thing that opens anonymous access, and under ADR-0121 D6 it now also requires an armed `rateLimit` (`enabled: true` — the key defaults to `false`, so a budget written without it meters nothing) or the stack refuses to publish. ⚠️ If you author endpoints in TypeScript, annotate them with `ApiEndpoint` — the AUTHOR state — so that omitting `authRequired` compiles: `const e: ApiEndpoint = { name, path, method, type, target }` is legal and is the safe shape this paragraph prescribes. `ApiEndpointParsed` is the POST-parse type (defaults materialized, ADR-0122), where `authRequired` is required — annotating a declaration with it forces you to write the key out, and being made to think about a key whose only unrecoverable value is `false` is the one thing this entry is trying to avoid (#5227). Hold a parse RESULT with `ApiEndpointParsed`; write declarations as `ApiEndpoint`. Grep every `apis:` entry for `authRequired: false` before you upgrade, delete the ones that were never meant to be public, and arm a budget on the ones that were. The path move is the mechanical-looking half and is still yours: ADR-0121 D1/D2 confine a declared path to your own namespace carve-out (`/api/v1/apps//…`), the namespace comes from an explicit `manifest.namespace` with no derivation fallback, and the subpath is the only part you name — rewriting it for you would silently change a URL third parties call." }, { "surface": "ui.widgetManifest / ui.widgetLifecycle / ui.widgetEvent / ui.widgetProperty / ui.widgetSource / ui.i18nObject / ui.pluralRule / ui.numberFormat / ui.dateFormat / ui.localeConfig (the widget-registration vocabulary of ui/widget.zod.ts, and the five doorless shapes of ui/i18n.zod.ts — 10 defs, 26 exported names)", @@ -1351,7 +1351,7 @@ "replacement": "the same declarations, re-read as LIVE HTTP routes: `path` moved under `/api/v1/apps//`, and every entry that declares `authRequired: false` re-confirmed as an intentionally anonymous endpoint carrying `rateLimit: { enabled: true, … }`", "migrationId": "declarative-apis-endpoints-live", "toMajor": 17, - "rationale": "This is the one protocol-17 entry that turns metadata ON rather than off, so read it as a SECURITY review item and not as a rename. Before 17 the declarative endpoint surface executed NOTHING: no route was mounted for a declared `path`, no matcher existed, and every key — `authRequired` included — parsed green and gated nothing (#4936, which refused a non-empty `apis:` outright for exactly that reason). Protocol 17 ships the executor (#5040) and narrows that refusal to a per-endpoint publish gate: an endpoint that PASSES the gate is mounted and serves real traffic as soon as the stack is published. So an `apis:` block written against an older major — or one restored from a pre-#4936 source, or authored from a doc that predates the refusal — changes meaning without changing a byte: what used to be inert documentation becomes an execution entry point into the data and automation pipelines. Nothing about that transition can be applied mechanically, because the judgment it needs is \"did the author of this endpoint mean for the internet to reach it?\" — and the one key where a wrong answer is unrecoverable is `authRequired`. Its schema default is `true`, so an omission is SAFE and needs no review; an EXPLICIT `authRequired: false` is the only thing that opens anonymous access, and under ADR-0121 D6 it now also requires an armed `rateLimit` (`enabled: true` — the key defaults to `false`, so a budget written without it meters nothing) or the stack refuses to publish. Grep every `apis:` entry for `authRequired: false` before you upgrade, delete the ones that were never meant to be public, and arm a budget on the ones that were. The path move is the mechanical-looking half and is still yours: ADR-0121 D1/D2 confine a declared path to your own namespace carve-out (`/api/v1/apps//…`), the namespace comes from an explicit `manifest.namespace` with no derivation fallback, and the subpath is the only part you name — rewriting it for you would silently change a URL third parties call." + "rationale": "This is the one protocol-17 entry that turns metadata ON rather than off, so read it as a SECURITY review item and not as a rename. Before 17 the declarative endpoint surface executed NOTHING: no route was mounted for a declared `path`, no matcher existed, and every key — `authRequired` included — parsed green and gated nothing (#4936, which refused a non-empty `apis:` outright for exactly that reason). Protocol 17 ships the executor (#5040) and narrows that refusal to a per-endpoint publish gate: an endpoint that PASSES the gate is mounted and serves real traffic as soon as the stack is published. So an `apis:` block written against an older major — or one restored from a pre-#4936 source, or authored from a doc that predates the refusal — changes meaning without changing a byte: what used to be inert documentation becomes an execution entry point into the data and automation pipelines. Nothing about that transition can be applied mechanically, because the judgment it needs is \"did the author of this endpoint mean for the internet to reach it?\" — and the one key where a wrong answer is unrecoverable is `authRequired`. Its schema default is `true`, so an omission is SAFE and needs no review; an EXPLICIT `authRequired: false` is the only thing that opens anonymous access, and under ADR-0121 D6 it now also requires an armed `rateLimit` (`enabled: true` — the key defaults to `false`, so a budget written without it meters nothing) or the stack refuses to publish. ⚠️ If you author endpoints in TypeScript, annotate them with `ApiEndpoint` — the AUTHOR state — so that omitting `authRequired` compiles: `const e: ApiEndpoint = { name, path, method, type, target }` is legal and is the safe shape this paragraph prescribes. `ApiEndpointParsed` is the POST-parse type (defaults materialized, ADR-0122), where `authRequired` is required — annotating a declaration with it forces you to write the key out, and being made to think about a key whose only unrecoverable value is `false` is the one thing this entry is trying to avoid (#5227). Hold a parse RESULT with `ApiEndpointParsed`; write declarations as `ApiEndpoint`. Grep every `apis:` entry for `authRequired: false` before you upgrade, delete the ones that were never meant to be public, and arm a budget on the ones that were. The path move is the mechanical-looking half and is still yours: ADR-0121 D1/D2 confine a declared path to your own namespace carve-out (`/api/v1/apps//…`), the namespace comes from an explicit `manifest.namespace` with no derivation fallback, and the subpath is the only part you name — rewriting it for you would silently change a URL third parties call." }, { "surface": "ui.widgetManifest / ui.widgetLifecycle / ui.widgetEvent / ui.widgetProperty / ui.widgetSource / ui.i18nObject / ui.pluralRule / ui.numberFormat / ui.dateFormat / ui.localeConfig (the widget-registration vocabulary of ui/widget.zod.ts, and the five doorless shapes of ui/i18n.zod.ts — 10 defs, 26 exported names)", diff --git a/packages/spec/src/api/endpoint.test.ts b/packages/spec/src/api/endpoint.test.ts index 022e20427d..8eb59b10e0 100644 --- a/packages/spec/src/api/endpoint.test.ts +++ b/packages/spec/src/api/endpoint.test.ts @@ -8,8 +8,10 @@ import { ApiMappingSchema, ApiEndpoint, } from './endpoint.zod'; +import type { ApiEndpointParsed } from './endpoint.zod'; import { RateLimitConfigSchema } from '../shared/http.zod'; import { HttpMethod } from './router.zod'; +import { getMetadataTypeSchema } from '../kernel/metadata-type-schemas'; describe('HttpMethod', () => { it('should accept valid HTTP methods', () => { @@ -443,3 +445,145 @@ describe('ApiEndpointSchema', () => { })).toThrow(); }); }); + +// --------------------------------------------------------------------------- +// #5384 — the shape is CLOSED. #5227 — the author state is expressible. +// --------------------------------------------------------------------------- + +/** + * A minimal endpoint that satisfies every required key and nothing else. + * + * Deliberately omits `authRequired`: that omission is the SAFE shape the + * protocol upgrade guide prescribes (`declarative-apis-endpoints-live`), and + * #5227 was filed because it used to be inexpressible in TypeScript. + */ +const AUTHORED_ENDPOINT = { + name: 'probe_endpoint', + path: '/api/v1/apps/probe/things', + method: 'GET', + type: 'object_operation', + target: 'probe_thing', +} as const; + +/** + * Every `unrecognized_keys` issue naming `key`, as the author meets it. + * + * `ApiEndpointSchema.safeParse` speaks in zod ISSUES, not in an ADR-0112 + * `{ code, status }` envelope — that envelope appears further out, where the + * `PUT /meta/api/:name` handler turns this verdict into a 422. So the named + * issue is what a rejection test at this layer must assert. `toThrow()` alone + * would pass on any failure at all, including the ones these cases exist to + * tell apart (refused by NAME with a prescription vs. refused for a missing + * required key). + */ +function unknownKeyIssues(result: ReturnType, key: string) { + if (result.success) return []; + return result.error.issues.filter( + (i) => i.code === 'unrecognized_keys' && (i as { keys?: string[] }).keys?.includes(key), + ); +} + +describe('#5384 — ApiEndpointSchema REJECTS undeclared keys', () => { + it('CONTROL — the authored endpoint parses, and the omitted `authRequired` defaults to true', () => { + // Without this the rejection cases below would also pass if the fixture + // were simply invalid, which proves nothing about strictness. + const ok = ApiEndpointSchema.safeParse(AUTHORED_ENDPOINT); + expect(ok.success, 'the CONTROL endpoint must parse').toBe(true); + expect(ok.data!.authRequired, 'the omission must still be fail-SAFE').toBe(true); + }); + + it('CONTROL — a loader-stamped ADR-0010 protection envelope still parses', () => { + // The reason the `...MetadataProtectionFields` spread is load-bearing now: + // undeclared, these would be REJECTED rather than dropped, and every + // registered `api` item the artifact loader stamps would stop parsing. + const stamped = ApiEndpointSchema.safeParse({ + ...AUTHORED_ENDPOINT, + _packageId: 'com.acme.things', + _provenance: 'package', + _lock: 'no-delete', + }); + expect(stamped.success).toBe(true); + }); + + it('refuses a plain undeclared key by name, naming the surface', () => { + const result = ApiEndpointSchema.safeParse({ ...AUTHORED_ENDPOINT, aKeyThatIsNotDeclared: 1 }); + expect(result.success).toBe(false); + const issues = unknownKeyIssues(result, 'aKeyThatIsNotDeclared'); + expect(issues).toHaveLength(1); + expect(issues[0]!.message).toContain('this API endpoint'); + expect(issues[0]!.message).toContain('aKeyThatIsNotDeclared'); + }); + + it.each([ + // The three typos the file header names as what the strip used to cost. + ['cacheTTL', 'cacheTtl'], + ['objectParam', 'objectParams'], + ['outputMappings', 'outputMapping'], + // The policy block, where a silent strip is worst. + ['auth', 'authRequired'], + ['requiresAuth', 'authRequired'], + ['rateLimiting', 'rateLimit'], + // Routing. + ['url', 'path'], + ['verb', 'method'], + ])('refuses `%s` and prescribes `%s`', (written, canonical) => { + const result = ApiEndpointSchema.safeParse({ ...AUTHORED_ENDPOINT, [written]: 1 }); + expect(result.success, `\`${written}\` still parses`).toBe(false); + const issues = unknownKeyIssues(result, written); + expect(issues, `\`${written}\` was not reported by name`).toHaveLength(1); + expect(issues[0]!.message).toContain(canonical); + }); + + it('refuses `namespace` with the ADR-0121 D2 wrong-layer prescription, not a rename', () => { + // The endpoint's namespace segment is DERIVED from `manifest.namespace` + // and has never been per-endpoint. Suggesting a rename here would steer the + // author at a key this surface refuses (ledger finding 7); the guidance + // entry suppresses the suggestion and points at the real layer instead. + const result = ApiEndpointSchema.safeParse({ ...AUTHORED_ENDPOINT, namespace: 'acme' }); + expect(result.success).toBe(false); + const issues = unknownKeyIssues(result, 'namespace'); + expect(issues).toHaveLength(1); + expect(issues[0]!.message).toContain('manifest.namespace'); + expect(issues[0]!.message).toContain('ADR-0121 D2'); + expect(issues[0]!.message).not.toContain('Did you mean'); + }); + + it('the registered `api` metadata type rejects it too — one contract, both doors', () => { + // The write path (`saveMetaItem` / `PUT /meta/api/:name`) parses through the + // registry, and the CLI parses through the stack root. They agreed while the + // shape was open (both accepted); the point of #5384 is that they now agree + // while REJECTING. The CLI half is pinned in + // `packages/cli/test/metadata-type-schema-gate.test.ts`, where `api` moved + // from NOT_YET_CLOSED into GATED_AT with this change. + const registered = getMetadataTypeSchema('api'); + expect(registered).toBeDefined(); + const result = registered!.safeParse({ ...AUTHORED_ENDPOINT, aKeyThatIsNotDeclared: 1 }); + expect(result.success).toBe(false); + expect( + result.error!.issues.some( + (i) => i.code === 'unrecognized_keys' && (i as { keys?: string[] }).keys?.includes('aKeyThatIsNotDeclared'), + ), + ).toBe(true); + }); +}); + +describe('#5227 — the author state is what `ApiEndpoint` denotes', () => { + it('omitting `authRequired` type-checks on the AUTHOR state and is the safe shape', () => { + // The compile-time half of this file's #5227 claim. `packages/spec` type + // checks its tests (`tsconfig.test.json`, AGENTS.md), so the annotation + // below is a real check rather than a phantom one — it fails the build if + // `ApiEndpoint` ever goes back to denoting the parsed state. + const authored: ApiEndpoint = { ...AUTHORED_ENDPOINT }; + expect(ApiEndpointSchema.safeParse(authored).success).toBe(true); + }); + + it('the PARSED state still requires it — the two names denote different types', () => { + // Control for the case above: if these were the same type, one of the two + // assertions could not hold, and #5227 would still be open. This is the + // exact TS2741 the issue reported, now attached to the name that should + // carry it. + // @ts-expect-error `authRequired` is required on the parsed state (ADR-0122). + const parsed: ApiEndpointParsed = { ...AUTHORED_ENDPOINT }; + expect(parsed).toBeDefined(); + }); +}); diff --git a/packages/spec/src/api/endpoint.zod.ts b/packages/spec/src/api/endpoint.zod.ts index 2c5e2688d9..0994aa26e4 100644 --- a/packages/spec/src/api/endpoint.zod.ts +++ b/packages/spec/src/api/endpoint.zod.ts @@ -3,6 +3,7 @@ import { z } from 'zod'; import { HttpMethod, RateLimitConfigSchema } from '../shared/http.zod'; import { MetadataProtectionFields } from '../kernel/metadata-protection.zod'; +import { strictObject } from '../shared/strict-object'; /** * API Mapping Schema @@ -16,45 +17,130 @@ export const ApiMappingSchema = lazySchema(() => z.object({ })); export type ApiMapping = z.input; +/** + * One prescription for every key of the metadata layer's STORED envelope. + * + * These are written by `MetadataManager.register` / `publishPackage` onto the + * stored ROW, never onto an authored declaration, and #5309 (PR #6576) made + * that separation explicit: `peelStoredEnvelope` takes them off before the body + * reaches this schema. So an author who meets this message wrote one by hand + * into a declaration — where it has never configured anything. + * + * Kept in sync by name with `STORED_ENVELOPE_KEYS` + * (`packages/metadata/src/stored-envelope.ts`). `packages/spec` holds no + * business logic and must not import from the metadata layer (Prime Directive + * #2), so the two lists are pinned equal from the metadata side + * (`stored-envelope.test.ts`) rather than shared as a value. + */ +const STORED_BOOKKEEPING_GUIDANCE = + 'This is the metadata layer\'s own storage bookkeeping, not endpoint vocabulary. It is written onto ' + + 'the stored ROW by `register` / `publishPackage` and peeled off before this schema sees a body ' + + '(#5309), so writing it on a declaration configures nothing. Remove it — publication state is ' + + 'managed by `objectstack publish`, not authored.'; + /** * API Endpoint Schema * Defines an external facing API contract. * - * ## Registered kind: envelope DECLARED, unknown keys still STRIPPED (#5271) + * ## Registered kind, CLOSED — and the ORDER it closed in is the record (#5384) * * `api` became a REGISTERED metadata kind in #5271 (part of #5206), which puts * this schema under the two invariants every registered kind is held to - * (`kernel/metadata-type-schemas.test.ts`). It satisfies one and is a measured - * exception to the other — and the reason is worth stating, because "close it - * like the other 23" is the obvious next edit and it does not work: + * (`kernel/metadata-type-schemas.test.ts`). It now satisfies both, and it took + * two steps in a deliberate order — kept here because a later sweep meeting a + * "cannot close, it parses stored rows" verdict elsewhere needs the refusal's + * reasoning, not just the result. * * - **The ADR-0010 protection envelope IS declared** (the spread at the bottom - * of the shape). The artifact loader stamps `_packageId` / `_provenance` on - * every registered item (`applyProtection`), and undeclared they were - * dropped on every parse — protection metadata lost on round-trip. + * of the shape, #5271). The artifact loader stamps `_packageId` / + * `_provenance` on every registered item (`applyProtection`), and undeclared + * they were dropped on every parse — protection metadata lost on round-trip. * - * - **Unknown keys are still stripped**, so `api` joins `view` on the #4001 - * campaign's `STILL_STRIP` list. This schema is not only an authoring - * surface: it is also what STORED rows are parsed with, by + * - **Unknown keys are REJECTED as of #5384.** Until then `api` sat beside + * `view` on the #4001 campaign's `STILL_STRIP` list, and the blocker was not + * this vocabulary: this schema is also what STORED rows were parsed with, by * `buildEndpointIndex` (`packages/metadata/src/endpoint-matcher.ts`) and by * `gateApiItemsForPublish` (`MetadataManager.publishPackage`). A stored row - * carries the metadata layer's own bookkeeping — `packageId` and `state`, - * written by `MetadataManager.register` / `publishPackage` and read back by - * `publishPackage`'s own package filter — which are NOT endpoint vocabulary. - * Closing this shape was tried and measured: every stored row fails with - * `unrecognized_keys: ['packageId', 'state']`, so the load-time backstop - * excludes it (its route answers 404) and the publish gate reports a schema - * error instead of the D6 verdict it exists to give. Exactly `view`'s shape - * of exception — one type name worn by both an authored document and a wire - * row — and the fix is to separate the stored envelope from the body at the - * metadata layer, not to teach this vocabulary two bookkeeping keys. + * carries the metadata layer's own bookkeeping — `packageId`, `state`, + * `version`, `published*` — which is NOT endpoint vocabulary. Closing the + * shape was tried and MEASURED at that point: every stored row failed with + * `unrecognized_keys: ['packageId', 'state']`, the load-time backstop + * excluded the endpoint (its route answered 404) and the publish gate + * reported a schema error in place of the ADR-0121 D6 verdict it exists to + * give — 11 tests in `packages/metadata` red. So the fix was made where the + * defect was: **#5309 / PR #6576 peeled the stored envelope off before the + * body parse** (`peelStoredEnvelope`, `packages/metadata/src/stored-envelope.ts`), + * which left exactly ONE red under a strict probe and made this closure a + * vocabulary change rather than a storage-layer compromise. Teaching this + * schema two bookkeeping keys to buy strictness would have made the + * authoring contract describe the storage layer — the trade this campaign + * refuses. + * + * What the strip was costing, now that it is gone: a `cacheTTL` / + * `outputMappings` / `objectParam` typo parsed green, published green, and the + * endpoint then served without the policy or projection its author wrote. That + * direction is fail-safe for `authRequired` alone (an unrecognized spelling + * left the default `true` standing); it was never fail-safe for the mapping, + * cache and rate-limit blocks, and AI-authored endpoint declarations are + * exactly where that class of typo is produced in bulk. + * + * ## Author state vs parsed state (#5227) * - * The cost of leaving it open is real and is filed rather than hidden: a - * `cacheTTL` / `outputMappings` / `objectParam` typo parses green, publishes - * green, and the endpoint then serves without the policy or projection its - * author wrote. + * `authRequired` is `.default(true)`, so the two shapes of this schema differ + * and both are named, per ADR-0122: **`ApiEndpoint` is the AUTHOR state** — + * omitting `authRequired` is expressible and is the SAFE spelling the protocol + * upgrade guide prescribes — while `ApiEndpointParsed` is the post-parse shape + * where the default has been materialized and the key is required. An author + * writing `const e: ApiEndpoint = { … }` can leave the key out; a consumer + * holding `ApiEndpointSchema.parse(...)` gets `ApiEndpointParsed`. Do NOT add + * an `ApiEndpointInput` alias: after ADR-0122 phase 2 it would denote exactly + * what `ApiEndpoint` denotes, which D3 forbids and `check:spec-parsed-alias` + * rule 4 rejects. */ -export const ApiEndpointSchema = z.object({ +export const ApiEndpointSchema = strictObject({ + surface: 'this API endpoint', + history: + 'Until #5384 closed this shape these were dropped silently — the endpoint still published and ' + + 'served, minus whatever the key was meant to configure (a `cacheTTL` / `objectParam` typo cost ' + + 'the whole policy or projection, with `os validate` green).', + aliases: { + // Policy block — the highest-consequence misses on this surface. + auth: 'authRequired', authentication: 'authRequired', requiresAuth: 'authRequired', + cacheTTL: 'cacheTtl', ttl: 'cacheTtl', cache: 'cacheTtl', + rateLimiting: 'rateLimit', throttle: 'rateLimit', + // Identity / routing. + id: 'name', url: 'path', route: 'path', endpoint: 'path', uri: 'path', + verb: 'method', httpMethod: 'method', + // Execution + projection. + handler: 'target', objectParam: 'objectParams', + inputMappings: 'inputMapping', outputMappings: 'outputMapping', + requestMapping: 'inputMapping', responseMapping: 'outputMapping', + title: 'summary', + }, + guidance: { + // [#5384] The wrong-layer pointer this closure exists to give. ADR-0121 D2: + // the namespace segment of a declared `path` is DERIVED from the stack's + // `manifest.namespace` and is never authored on the endpoint. A `namespace` + // key here parsed green and gated nothing, while the publish gate went on + // reading the manifest — so an author who "fixed" a namespace mismatch here + // changed nothing at all. `publish-endpoint-gate.test.ts` pins that the gate + // does not believe this key; it is now refused by name before the gate runs. + namespace: + '`namespace` is not an endpoint key. Under ADR-0121 D2 the namespace segment of `path` comes ' + + "from the stack's `manifest.namespace` and has no per-endpoint override — declare it there. " + + 'Only the subpath of `/api/v1/apps//` is yours to name here.', + // [#5309 / PR #6576] The metadata layer's own storage bookkeeping. These + // reach this schema on the STORED-row paths and are peeled by + // `peelStoredEnvelope` before the body parse, so meeting one here means it + // was hand-authored into a declaration — where it configures nothing. + packageId: STORED_BOOKKEEPING_GUIDANCE, + state: STORED_BOOKKEEPING_GUIDANCE, + version: STORED_BOOKKEEPING_GUIDANCE, + publishedAt: STORED_BOOKKEEPING_GUIDANCE, + publishedBy: STORED_BOOKKEEPING_GUIDANCE, + publishedDefinition: STORED_BOOKKEEPING_GUIDANCE, + }, +}, { /** Identity */ name: z.string().regex(/^[a-z_][a-z0-9_]*$/).describe('Unique endpoint ID'), path: z.string().regex(/^\//).describe( @@ -93,6 +179,9 @@ export const ApiEndpointSchema = z.object({ // ADR-0010 — runtime protection envelope (internal — set by the loader). // `api` is a registered metadata kind as of #5271, so the artifact loader // stamps these on every item; undeclared they were dropped on every parse. + // Load-bearing twice over since #5384: with the shape closed, an undeclared + // protection key would now be REJECTED rather than dropped, so the spread is + // what keeps a loader-stamped item parsing at all. ...MetadataProtectionFields, }); diff --git a/packages/spec/src/kernel/metadata-type-schemas.test.ts b/packages/spec/src/kernel/metadata-type-schemas.test.ts index 6a02944923..5f3a9c0506 100644 --- a/packages/spec/src/kernel/metadata-type-schemas.test.ts +++ b/packages/spec/src/kernel/metadata-type-schemas.test.ts @@ -254,33 +254,45 @@ describe('registered metadata types', () => { * exists to draw, arriving here as the campaign's final answer rather than as an * exception to it. * - * ## `api` arrives (2026-08-04, #5271) with the SAME distinction, not a new one + * ## `api` arrived (2026-08-04, #5271) and has now LEFT this list (#5384) * - * `api` joined the registry after the campaign ended, and it lands on this list - * for `view`'s reason wearing different clothes: `ApiEndpointSchema` is not only - * an authoring surface — it is what STORED rows are parsed with, by - * `buildEndpointIndex` (`packages/metadata/src/endpoint-matcher.ts`) and by + * Kept as history, because the two entries looked identical and only one of + * them was the end state — and telling them apart is what this list is for. + * + * `api` joined the registry after the campaign ended and landed here wearing + * `view`'s clothes: `ApiEndpointSchema` is not only an authoring surface — it + * is what STORED rows were parsed with, by `buildEndpointIndex` + * (`packages/metadata/src/endpoint-matcher.ts`) and by * `MetadataManager.publishPackage`'s `gateApiItemsForPublish`. A stored row - * carries the metadata layer's own bookkeeping (`packageId`, `state` — written - * by `register` / `publishPackage`, and read back by `publishPackage`'s package - * filter), which is not endpoint vocabulary. + * carries the metadata layer's own bookkeeping (`packageId`, `state`, `version`, + * `published*` — written by `register` / `publishPackage`, and read back by + * `publishPackage`'s package filter), which is not endpoint vocabulary. * - * This was MEASURED, not assumed. Closing the shape with `strictObject` turns + * That was MEASURED, not assumed. Closing the shape with `strictObject` turned * every stored row into `unrecognized_keys: ['packageId', 'state']`: the - * load-time backstop then excludes the endpoint (its route answers 404) and the - * publish gate reports a schema error in place of the ADR-0121 D6 verdict it - * exists to give — 10 tests in `packages/metadata` go red, which is what - * surfaced it. So the debt is real and it is NOT in this vocabulary: the fix is - * to separate the stored envelope from the body at the metadata layer, filed - * separately, after which `api` comes off this list. Teaching `ApiEndpointSchema` - * two bookkeeping keys to buy strictness would make the authoring contract - * describe the storage layer, which is the trade this campaign refuses. + * load-time backstop then excluded the endpoint (its route answered 404) and the + * publish gate reported a schema error in place of the ADR-0121 D6 verdict it + * exists to give — 11 tests in `packages/metadata` went red, which is what + * surfaced it. The conclusion drawn at the time was the one that held up: the + * debt was real and it was NOT in this vocabulary. + * + * **#5309 (PR #6576) paid it at the right layer** — `peelStoredEnvelope` + * (`packages/metadata/src/stored-envelope.ts`) takes the envelope off before the + * body parse, at both parse sites — and under a strict probe on that tree + * exactly ONE test remained red, a fixture planting an authored `namespace` key + * that the closed shape now refuses by name. So `api` closed at **#5384** as an + * ordinary #4001 conversion, and it is off this list. `ApiEndpointSchema` never + * learned a bookkeeping key: teaching it two would have made the authoring + * contract describe the storage layer, which is the trade this campaign refuses. * - * So: `view` is the end state; `api` is tracked debt with a named owner. Both - * are here for the same underlying reason — one type name serving both an - * authored document and a wire row. + * So: `view` is the end state, and `api` was tracked debt with a named owner + * that got paid. Both were here for the same underlying reason — one type name + * serving both an authored document and a wire row — and the difference is that + * `view`'s open members are wire shapes with nowhere else to live, while `api`'s + * envelope had a layer that could own it. A future entry on this list should be + * asked which of the two it is before it is accepted as permanent. */ -const STILL_STRIP = new Set(['view', 'api']); +const STILL_STRIP = new Set(['view']); /** The registered schema's own top-level posture: `.strict()` sets a `never` catchall. */ function topLevelPosture(schema: unknown, depth = 0): 'strict' | 'strip' | null { @@ -368,7 +380,16 @@ describe('#4001 — registered-type closure is derived, not tallied', () => { // body via `capabilityRowFields`), which is exactly the property `api` // lacks. So the closed count moves with the total, 23 → 24, and // `STILL_STRIP` does not grow. - expect(closed.length).toBe(24); + // + // 24 → 25 CLOSED on 2026-08-08: `api` closed (#5384). The registered total + // does not move — only the posture does. The property `capability` had and + // `api` lacked above turned out not to be a permanent difference: #5309 + // (PR #6576) gave the stored envelope a layer of its own + // (`peelStoredEnvelope`), so nothing re-parses a stored `api` row through + // this schema either, and the conversion became an ordinary #4001 one. + // `STILL_STRIP` shrinks to `view` alone, which IS the end state — its open + // members are wire shapes with nowhere else to live (see that list's note). + expect(closed.length).toBe(25); expect(types.length).toBe(26); }); }); diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index 106e3ff569..35dda5e86c 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -1852,7 +1852,15 @@ const step17: MigrationStep = { + '`authRequired: false` is the only thing that opens anonymous access, and under ' + 'ADR-0121 D6 it now also requires an armed `rateLimit` (`enabled: true` — the key ' + 'defaults to `false`, so a budget written without it meters nothing) or the stack ' - + 'refuses to publish. Grep every `apis:` entry for `authRequired: false` before you ' + + 'refuses to publish. ⚠️ If you author endpoints in TypeScript, annotate them with ' + + '`ApiEndpoint` — the AUTHOR state — so that omitting `authRequired` compiles: ' + + '`const e: ApiEndpoint = { name, path, method, type, target }` is legal and is the ' + + 'safe shape this paragraph prescribes. `ApiEndpointParsed` is the POST-parse type ' + + '(defaults materialized, ADR-0122), where `authRequired` is required — annotating a ' + + 'declaration with it forces you to write the key out, and being made to think about a ' + + 'key whose only unrecoverable value is `false` is the one thing this entry is trying ' + + 'to avoid (#5227). Hold a parse RESULT with `ApiEndpointParsed`; write declarations ' + + 'as `ApiEndpoint`. Grep every `apis:` entry for `authRequired: false` before you ' + 'upgrade, delete the ones that were never meant to be public, and arm a budget on the ' + 'ones that were. The path move is the mechanical-looking half and is still yours: ' + 'ADR-0121 D1/D2 confine a declared path to your own namespace carve-out ' From ac31c383b35753c6cca3b33428693cd323e3681a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 8 Aug 2026 22:44:55 +0000 Subject: [PATCH 2/4] fix(spec): `package` was missing from the endpoint stored-bookkeeping guidance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Caught by the drift pin added in the same change, on its first run: `STORED_ENVELOPE_KEYS` has SEVEN entries and the guidance table listed six. This is the drift the pin exists for, and it is silent in the direction that matters — `package` is peeled correctly on the stored paths, so every other test stays green, while an author who hand-writes it into a declaration gets the generic "unrecognized key" instead of the wrong-layer prescription. A rejection arriving without its upgrade is the ledger's finding 7. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011sGk4SKHqGRgmmqUok1P8M --- packages/spec/src/api/endpoint.zod.ts | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/packages/spec/src/api/endpoint.zod.ts b/packages/spec/src/api/endpoint.zod.ts index 0994aa26e4..4a0380728b 100644 --- a/packages/spec/src/api/endpoint.zod.ts +++ b/packages/spec/src/api/endpoint.zod.ts @@ -133,6 +133,15 @@ export const ApiEndpointSchema = strictObject({ // reach this schema on the STORED-row paths and are peeled by // `peelStoredEnvelope` before the body parse, so meeting one here means it // was hand-authored into a declaration — where it configures nothing. + // + // ⚠️ All SEVEN of `STORED_ENVELOPE_KEYS`, and the completeness is pinned + // rather than eyeballed: `stored-envelope.test.ts` walks that exported list + // and asserts each key gets this prescription. The first run of that pin + // caught `package` missing from this table — the drift is silent in the + // direction that matters (a key peeled correctly on the stored paths, so + // every other test stays green, while a hand-authoring author gets the + // generic "unrecognized key" instead of the upgrade). + package: STORED_BOOKKEEPING_GUIDANCE, packageId: STORED_BOOKKEEPING_GUIDANCE, state: STORED_BOOKKEEPING_GUIDANCE, version: STORED_BOOKKEEPING_GUIDANCE, From 6794b29a0d34e3eeb6c264782a7d6cfc2538eb24 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 00:06:03 +0000 Subject: [PATCH 3/4] chore(changeset): answer the ADR-0087 disposition question in writing (#6148) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gate does not decide whether an entry is needed; it requires the question be answered. Answered `not-required (no-migration-prescription)`: this change retires no key, only the unknown-key posture moves, and an undeclared key was never honoured — so no stored shape carries it and nothing that ever worked becomes invalid. The schema rejection is the upgrade channel and is more specific than a ledger line could be. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011sGk4SKHqGRgmmqUok1P8M --- .changeset/api-endpoint-schema-strict.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.changeset/api-endpoint-schema-strict.md b/.changeset/api-endpoint-schema-strict.md index f703a605d3..5e4652b8a7 100644 --- a/.changeset/api-endpoint-schema-strict.md +++ b/.changeset/api-endpoint-schema-strict.md @@ -2,6 +2,8 @@ "@objectstack/spec": minor --- + + `ApiEndpointSchema` rejects undeclared keys (#5384), and the author-state type is named on the upgrade path (#5227) `api` became a registered metadata type at #5312, which made From a2f7772fbee34a80d0b6ca6d8cb12a05ec2ca3ec Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 00:14:59 +0000 Subject: [PATCH 4/4] chore(changeset): state the ADR-0087 disposition without rewrite-arrow syntax MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The marker's own category name supplies the word 'migration', and the two worked examples supplied the arrow — so the disposition line matched the gate's framed-line rewrite-prescription detector and contradicted itself. Same reasoning, stated without FROM/TO arrow syntax; the per-key prescriptions live in the schema's aliases/guidance, which is where an author meets them. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011sGk4SKHqGRgmmqUok1P8M --- .changeset/api-endpoint-schema-strict.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.changeset/api-endpoint-schema-strict.md b/.changeset/api-endpoint-schema-strict.md index 5e4652b8a7..bffc72cf93 100644 --- a/.changeset/api-endpoint-schema-strict.md +++ b/.changeset/api-endpoint-schema-strict.md @@ -2,7 +2,7 @@ "@objectstack/spec": minor --- - + `ApiEndpointSchema` rejects undeclared keys (#5384), and the author-state type is named on the upgrade path (#5227)