Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .changeset/datasource-external-inert-keys-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
"@objectstack/spec": major
"@objectstack/example-showcase": patch
---

feat(spec)!: retire `external.label` and `external.requirePermission` (#4583 batch D)

Two keys on the federation block, both read by nothing.

**`external.label`** — nothing rendered the federation block's own label. Setup →
Datasources renders the datasource's **top-level** `label`, which every datasource already
has, so this was a second display name that never displayed. The showcase example declared
both; it now declares only the one that shows.

**`external.requirePermission`** — no authorization check ever consulted it. A permission
named here gated nothing: access to a federated datasource's data is governed by the
ordinary object permission sets and RLS, exactly as for a managed datasource. Naming a
permission that is never required is the false-compliance shape ADR-0049 exists to remove
— it reads like an access control and is one only in the author's head.

FROM → TO: delete `external.label` (use the top-level `label`); delete
`external.requirePermission` and grant or withhold the object permissions instead.
`os migrate meta --from 16` removes both automatically (conversion
`datasource-inert-blocks-removed`).

With these, the `datasource` liveness ledger reaches **zero dead properties** — down from
the 20 it was seeded with in #4487, the highest dead ratio of any governed type.
20 changes: 20 additions & 0 deletions .changeset/datasource-health-check-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
"@objectstack/spec": major
---

feat(spec)!: retire `datasource.healthCheck` — no probe loop ever existed (#4583 batch C)

Three keys — `enabled`, `intervalMs`, `timeoutMs` — declared, strict-guarded, read by
nothing. No health-check loop was ever scheduled, so `enabled: true` enabled nothing and
the two timeouts bounded nothing.

Connection liveness is probed **on demand** through the driver handle's `ping()` /
`checkHealth()`, which the datasource admin service calls for "Test connection". That is
the mechanism — it needs no configuration here and never read this block.

Note what it is NOT to be confused with: `external.validation.checkIntervalMs` is the one
recurring datasource timer, and it checks **schema drift** on a federated datasource, not
connection liveness. It is unaffected.

FROM → TO: delete the block. `os migrate meta --from 16` removes it automatically
(conversion `datasource-inert-blocks-removed`).
21 changes: 21 additions & 0 deletions .changeset/datasource-retry-policy-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
"@objectstack/spec": major
---

feat(spec)!: retire `datasource.retryPolicy` — nothing ever retried on it (#4583 batch B)

Four keys — `maxRetries`, `baseDelayMs`, `maxDelayMs`, `backoffMultiplier` — declared,
strict-guarded, and read by no connect or query path. Connection failure is handled by
the boot policy in the datasource connection service (degraded boot, or `bootCritical`
fail-fast); nothing retries on a schedule, so setting `maxRetries: 5` changed nothing.

**Do not "fix" this by renaming keys.** `hook.retryPolicy` and `job.retryPolicy` ARE
enforced — but they are a different key on a different type, and they spell the delay
`backoffMs`, not `baseDelayMs`. That very inconsistency is the evidence nothing read the
datasource one: no code in the repo reads both spellings. Moving these values onto a hook
or a job only makes sense if you actually want that hook or job retried.

FROM → TO: delete the block. `os migrate meta --from 16` removes it automatically
(conversion `datasource-inert-blocks-removed`). `DatasourceSchema` is `.strict()`, so a
leftover `retryPolicy` is a loud rejection carrying this prescription — never a silent
strip.
6 changes: 1 addition & 5 deletions content/docs/references/data/datasource.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,12 @@ const result = DatasourceSchema.parse(data);
| **driver** | `string` | ✅ | Underlying driver type |
| **config** | `Record<string, any>` | ✅ | Driver specific configuration |
| **pool** | `{ min: number; max: number; idleTimeoutMillis: number; connectionTimeoutMillis: number }` | optional | Connection pool settings |
| **healthCheck** | `{ enabled: boolean; intervalMs: number; timeoutMs: number }` | optional | Datasource health check configuration |
| **ssl** | `{ enabled: boolean; rejectUnauthorized: boolean; ca?: string; cert?: string; … }` | optional | SSL/TLS configuration for secure database connections |
| **retryPolicy** | `{ maxRetries: number; baseDelayMs: number; maxDelayMs: number; backoffMultiplier: number }` | optional | Connection retry policy for transient failures |
| **description** | `string` | optional | Internal description |
| **active** | `boolean` | ✅ | Is datasource enabled |
| **autoConnect** | `boolean` | ✅ | Force a live driver connection at boot even when managed + unrouted (ADR-0062 D2). |
| **schemaMode** | `Enum<'managed' \| 'external' \| 'validate-only'>` | ✅ | Schema ownership mode |
| **external** | `{ label?: string; allowedSchemas?: string[]; allowWrites: boolean; validation: object; … }` | optional | External datasource federation settings (schemaMode != "managed") |
| **external** | `{ allowedSchemas?: string[]; allowWrites: boolean; validation: object; credentialsRef?: string; … }` | optional | External datasource federation settings (schemaMode != "managed") |
| **origin** | `Enum<'code' \| 'runtime'>` | ✅ | Datasource provenance (server-managed, read-only) |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
Expand Down Expand Up @@ -82,13 +80,11 @@ External datasource federation settings (schemaMode != "managed")

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **label** | `string` | optional | Display label, e.g. "Snowflake — ANALYTICS / PROD" |
| **allowedSchemas** | `string[]` | optional | Whitelist of remote schemas/databases that may be exposed. |
| **allowWrites** | `boolean` | ✅ | Global write gate. Individual objects must also opt in via object.external.writable. |
| **validation** | `{ onMismatch: Enum<'fail' \| 'warn' \| 'ignore'>; checkOnBoot: boolean; checkIntervalMs?: number }` | ✅ | Boot/drift validation policy |
| **credentialsRef** | `string` | optional | Reference into the secrets store; never inline credentials. |
| **queryTimeoutMs** | `number` | ✅ | Hard cap on per-query execution time. |
| **requirePermission** | `string` | optional | Optional convenience: gate the entire datasource behind a single role. |


---
Expand Down
4 changes: 2 additions & 2 deletions docs/audits/2026-07-unknown-key-strictness-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | |
| `sharing.zod.ts` | 2 | authorable (p) | public-sharing config |

### `data/` — 164 sites
### `data/` — 162 sites

| File | Sites | Class | Note |
|---|---|---|---|
Expand All @@ -465,7 +465,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `field.zod.ts` | 11 | authorable | partially strict |
| `filter.zod.ts` / `query.zod.ts` | 11+5 | open | query dialect — user data flows through; validated semantically elsewhere. `query.zod.ts` dropped one site in #4196: `FieldNodeSchema`'s nested-select object form was declared-but-inert and narrowed to `z.string()`, so the union's second member is gone. Four more left in #4286 with the `joins`/`windowFunctions` removals: `JoinNodeBaseSchema`, `WindowFunctionNodeSchema`, and `WindowSpecSchema`'s two blocks (outer + `frame`) were deleted with their clusters. Class unchanged |
| `driver-nosql.zod.ts` / `driver.zod.ts` / `driver-sql.zod.ts` | 10+9+2 | wire | driver capability contracts |
| `datasource.zod.ts` | 8 | authorable | **strict as of #4001 data step** — all 8: `DatasourceSchema` (+ `pool` / `healthCheck` / `ssl` / `retryPolicy`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DriverDefinitionSchema`. `config` stays `z.record` **at this level** by construction (per-driver shapes), but is no longer unchecked: **#4410** made `DatasourceSchema`'s refinement parse it against the contract for the declared driver (`driver/config-registry.zod.ts`), so the openness here is a shape this level cannot express rather than the absence of one. This row used to add "the driver's own `configSchema` validates them", which was false until #4410 landed the parse site it names. #4410 extended the same parse to each `readReplicas` entry; **#4468 retired that key** — no driver ever opened a replica connection and no query path splits reads from writes, so the entries were being checked against a contract nothing would apply. Strictness makes a dropped key loud; it cannot make a slot live, and a *precisely validated* dead slot is the more convincing lie | **#4583 dropped the ninth site**: `DatasourceCapabilities` is gone — eleven flags no code read, on a block whose strictness was the clearest case of this row's own closing sentence. `readOnly` in particular was *precisely validated* and completely inert, and had been relocated twice (#4410, #4465) toward somewhere it might be enforced; the shipped CRM example called a datasource a read replica on the strength of it while writes went through. Class unchanged
| `datasource.zod.ts` | 6 | authorable | **strict as of #4001 data step** — all 6: `DatasourceSchema` (+ `pool` / `ssl`), `ExternalDatasourceSettingsSchema` (+ `validation`), `DriverDefinitionSchema`. **#4583 B/C dropped two more sites**: the `healthCheck` and `retryPolicy` blocks are gone — nothing scheduled a probe and nothing retried, so their strictness was validating a shape no code consumed. `config` stays `z.record` **at this level** by construction (per-driver shapes), but is no longer unchecked: **#4410** made `DatasourceSchema`'s refinement parse it against the contract for the declared driver (`driver/config-registry.zod.ts`), so the openness here is a shape this level cannot express rather than the absence of one. This row used to add "the driver's own `configSchema` validates them", which was false until #4410 landed the parse site it names. #4410 extended the same parse to each `readReplicas` entry; **#4468 retired that key** — no driver ever opened a replica connection and no query path splits reads from writes, so the entries were being checked against a contract nothing would apply. Strictness makes a dropped key loud; it cannot make a slot live, and a *precisely validated* dead slot is the more convincing lie | **#4583 dropped the ninth site**: `DatasourceCapabilities` is gone — eleven flags no code read, on a block whose strictness was the clearest case of this row's own closing sentence. `readOnly` in particular was *precisely validated* and completely inert, and had been relocated twice (#4410, #4465) toward somewhere it might be enforced; the shipped CRM example called a datasource a read replica on the strength of it while writes went through. Class unchanged
| `driver/memory.zod.ts` / `driver/mongo.zod.ts` / `driver/postgres.zod.ts` | 6+1+1 | authorable | The per-driver shapes for the `config` slot — what an author actually writes under `datasource.config` (`host`, `port`, `filename`). **Undeclared here until the coverage walk went recursive** (see below): a subdirectory was invisible to the gate, so these sites sat outside the map while the map reported full coverage. **Strict as of #4410**, which is also what unblocked them: this row previously read "strictness here would enforce nothing" because nothing parsed `datasource.config` against these schemas and both `*DriverSpec.configSchema` literals were `{}`. Now `DatasourceSchema` parses `config` against them, and the same schemas project onto `configSchema` and onto the Studio connection form. (#4410 also ran the parse over each `readReplicas` entry; #4468 retired that key outright — see the row above.) `postgres.zod.ts` drops a site: its `ssl` was a `boolean | {ca, cert, key, …}` union, and the object arm is gone — certificates now live in the datasource-level `ssl` block (declared, strict, and until #4410 read by nobody), leaving `config.ssl` as the on/off shorthand. That narrowing is forced by the same projection: the Studio form renders anything that is not boolean/enum/number as a TEXT INPUT, so a union here would have produced a wizard whose every `ssl` value the new gate rejects. `memory.zod.ts` keeps 6 but loses two KEYS — `indexes` / `maxRecordsPerObject`, which `InMemoryDriverConfig` has no field for, removed under ADR-0049 rather than blessed by the new gate |
| `driver/mysql.zod.ts` / `driver/sqlite.zod.ts` | 1+2 | authorable | The rest of the `config` contract, added by #4410. `mysql.zod.ts` and `sqlite.zod.ts` (sqlite + sqlite-wasm) are shapes that **never existed** — both driver ids were offered by the connection form and buildable by the shared factory, with no config contract anywhere, so `driver: 'sqlite'` + a misspelled `filename` was an ephemeral `:memory:` database reported as configured. All three sites strict, same error factory as the rest of the campaign. (Their sibling `driver/common.zod.ts` holds shared enums and prescription strings and has no `z.object(` site, so the coverage gate skips it) |
| `analytics.zod.ts` | 8 | mixed (p) | |
Expand Down
1 change: 1 addition & 0 deletions docs/protocol-upgrade-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,7 @@ The `script` flow node converges on its one real path (#4343). It had four ways
| `flow-node-wait-timeout-keys-removed` | `flow.node.waitEventConfig` | waitEventConfig keys 'timeoutMs' (→ 'timerDuration', stringified — its only reader used it as the duration) and 'onTimeout' (removed — zero readers, so no timeout ever fired) (#4158) | retired — `migrate meta` only |
| `datasource-read-replicas-removed` | `datasource.readReplicas` | datasource key 'readReplicas' removed (#4468 — no driver opened a replica connection and no query path splits reads from writes; front replicas behind one endpoint and point `config` at it) | retired — `migrate meta` only |
| `datasource-capabilities-removed` | `datasource.capabilities` | datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only) | retired — `migrate meta` only |
| `datasource-inert-blocks-removed` | `datasource.retryPolicy / datasource.healthCheck / datasource.external.label / datasource.external.requirePermission` | datasource keys 'retryPolicy'/'healthCheck' and external 'label'/'requirePermission' removed (#4583 — nothing retried, nothing probed on a schedule, and the federation label/permission were read by nobody) | retired — `migrate meta` only |
| `flow-node-script-branch-keys-removed` | `flow.node.script.config.actionType / flow.node.script.config.template / flow.node.script.config.recipients / flow.node.script.config.variables / flow.node.script.config.script` | script flow-node config keys 'actionType' (→ 'function' when it was shorthand for one; otherwise removed — 'email'/'slack' were logger-backed stubs that delivered nothing), plus 'template' / 'recipients' / 'variables' (fed those stubs) and 'script' (inline JS the runtime never executed) (#4343) | retired — `migrate meta` only |

### Semantic (delegated to you, with acceptance criteria)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ export const ShowcaseExternalDatasource = defineDatasource({
// same place the fixture writes it. Sits next to the managed standalone.db.
config: { filename: '.objectstack/data/showcase_external.db' },
external: {
label: 'External Analytics DB — read-only federation demo (ADR-0015)',
// `external.label` was removed in #4583 — nothing read the federation
// block's own label; the top-level `label` above is what Setup renders.
allowWrites: false,
validation: { onMismatch: 'warn', checkOnBoot: true },
},
Expand Down
29 changes: 14 additions & 15 deletions packages/cli/src/utils/lint-liveness-properties.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,28 +195,27 @@ describe('lintLivenessProperties', () => {
// the type that most needed it: 20 of its 43 props have no runtime consumer,
// and until #4487 nothing told an author so.

it('warns on the dead datasource blocks that remain — healthCheck / retryPolicy (#4487)', () => {
// `capabilities` left this list in #4583: the block was REMOVED from the
// schema, so an author who writes it now gets a hard parse rejection with a
// prescription — a stronger signal than a lint warning, and the reason its
// ledger rows are gone rather than flipped. healthCheck / retryPolicy are
// still authorable and still dead (batches B and C of #4583).
it('no longer warns on ANY datasource block — the whole dead surface is gone (#4583)', () => {
// This assertion has now inverted twice, and the direction of travel is the
// point. It began (#4487) asserting warnings on capabilities/healthCheck/
// retryPolicy; batch A removed `capabilities`, so it narrowed to the other
// two; batches B/C/D removed those as well. Every one of the twenty dead
// datasource properties is now a hard parse rejection carrying its own
// prescription — strictly stronger than an advisory lint warning, which is
// why their ledger rows are deleted rather than flipped.
//
// Kept (rather than deleted) as a REGRESSION GUARD: it runs against the
// real shipped ledger, so re-introducing a dead+authorWarn datasource
// property fails here rather than shipping quietly.
const findings = lintLivenessProperties({
datasources: [{
name: 'warehouse',
label: 'Warehouse',
driver: 'postgres',
config: { host: 'db.internal', database: 'analytics' },
healthCheck: { enabled: true, intervalMs: 30000 },
retryPolicy: { maxRetries: 5, baseDelayMs: 1000 },
}],
});
const msgs = paths(findings);
expect(msgs.some((m) => m.includes('healthCheck.enabled'))).toBe(true);
expect(msgs.some((m) => m.includes('healthCheck.intervalMs'))).toBe(true);
expect(msgs.some((m) => m.includes('retryPolicy.maxRetries'))).toBe(true);
expect(msgs.some((m) => m.includes('retryPolicy.baseDelayMs'))).toBe(true);
// The removed block must no longer be reported by the lint at all.
expect(msgs.some((m) => m.includes('capabilities'))).toBe(false);
expect(findings).toEqual([]);
});

// The entry the whole audit was worth doing for. `capabilities.readOnly` read
Expand Down
4 changes: 0 additions & 4 deletions packages/spec/authorable-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -3238,12 +3238,10 @@
"data/Datasource:description",
"data/Datasource:driver",
"data/Datasource:external",
"data/Datasource:healthCheck",
"data/Datasource:label",
"data/Datasource:name",
"data/Datasource:origin",
"data/Datasource:pool",
"data/Datasource:retryPolicy",
"data/Datasource:schemaMode",
"data/Datasource:ssl",
"data/Dimension:description",
Expand Down Expand Up @@ -3392,9 +3390,7 @@
"data/ExternalDatasourceSettings:allowWrites",
"data/ExternalDatasourceSettings:allowedSchemas",
"data/ExternalDatasourceSettings:credentialsRef",
"data/ExternalDatasourceSettings:label",
"data/ExternalDatasourceSettings:queryTimeoutMs",
"data/ExternalDatasourceSettings:requirePermission",
"data/ExternalDatasourceSettings:validation",
"data/ExternalFieldMapping:defaultValue",
"data/ExternalFieldMapping:readonly",
Expand Down
Loading
Loading