Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .changeset/datasource-config-key-alias-conversion.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
"@objectstack/spec": minor
"@objectstack/service-datasource": minor
---

feat(spec,service-datasource): graduate the driver factory's four legacy `datasource.config` `??` fallbacks into an ADR-0087 conversion (#4456)

`createDefaultDatasourceDriverFactory` still carried four undeclared read-side
`??` fallbacks that predate the #4410 config gate: sqlite `file`/`database`
(canonical `filename`), postgres/mysql `connectionString` (canonical `url`),
postgres/mysql/mongo `user` (canonical `username`), and mongo `uri` (canonical
`url`). They were never part of the contract — no schema, form, doc or example
ever named them — and they kept working only because the reader was lenient
(AGENTS.md Prime Directive #12 debt).

**FROM → TO, applied automatically at load** by the new conversion entry
`datasource-config-driver-key-aliases` (retired-from-load-path; replayed over
stored `sys_metadata` rows by `applyConversionsToStoredItem` and by
`os migrate meta`):

- sqlite / sqlite-wasm: `config.file` / `config.database` → `config.filename`
- postgres / mysql: `config.connectionString` → `config.url`, `config.user` → `config.username`
- mongo: `config.uri` → `config.url`, `config.user` → `config.username`

The mapping is driver-aware — `database` renames only under sqlite, where it
aliased the file path; for postgres/mysql/mongo it is a canonical key and is
untouched. A canonical key already present wins; the legacy alias is left
shadowed (the factory's `??` precedence, preserved).

**Behaviour change (the deletion):** the factory now reads exactly one spelling
per key. A `DatasourceConnectionSpec` handed to the factory *directly* with a
legacy spelling is no longer honoured — authored metadata was already rejected
by the per-driver zod gate with a rename hint (#4410), and stored runtime
datasource rows are canonicalized at every rehydration seam (including the
`sys_metadata` restore path in `DatasourceAdminServicePlugin`, which now
replays the full conversion chain), so no supported path still produces the
legacy shape. One-line fix for hand-built specs: use the canonical key from
the table above.
12 changes: 5 additions & 7 deletions content/docs/references/data/driver-sqlite.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,15 @@ fell back to `:memory:`, and their data vanished on restart with every signal

saying the datasource was configured.

`file` and `database` are a different case — the factory reads them as
`file` and `database` once also worked, purely because the factory read them

undeclared `??` fallbacks, so they happened to work while being documented
as undeclared `??` fallbacks. That tolerance has graduated into the declared

nowhere. They are named as renames here rather than blessed: one strict
ADR-0087 conversion `datasource-config-driver-key-aliases` (#4456): stored

contract beats a spelling that works only because a reader is lenient
rows are rewritten to `filename` at load, the factory reads one spelling,

(AGENTS.md Prime Directive #12). The factory keeps its tolerance for records

already persisted that way; no new one can be authored.
and authoring rejects both with the rename hint below.

<Callout type="info">
**Source:** `packages/spec/src/data/driver/sqlite.zod.ts`
Expand Down
3 changes: 3 additions & 0 deletions docs/protocol-upgrade-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,8 @@ The same kind of retirement covers `wait`'s timeout pair (#4158). `waitEventConf

Closing the same audit on the data side, `datasource.readReplicas` is removed (#4468). It described replica connections nothing ever opened: `ConnectableDatasource` and `DatasourceConnectionSpec` carry no replicas field, the driver factory never reads the key, and no query path distinguishes a read from a write — read/write splitting does not exist in the platform, so every statement always went to the primary. A lossless delete with no target to move to; front replicas behind one endpoint (pgpool, ProxySQL, an RDS reader endpoint) and point `config` at it. Notable as the case that shows how a key gets MORE convincing as it stays dead: #4410, closing the datasource-config gap, taught the schema to validate each replica entry against the declared driver's config contract, so sources written in between carry replica blocks that were genuinely checked — precise hosts, correct port types, typos rejected. Precision applied to an inert slot reads as evidence the slot is live, which is why ADR-0049 asks for a consumer rather than for rigor. Retired from the load path with the rest of the keys that misdescribed themselves.

The datasource close-out also graduates the four legacy `datasource.config` spellings the shared driver factory still tolerated via undeclared read-side `??` fallbacks (#4456, the #4410 follow-up): sqlite `file`/`database` (use `filename`), postgres/mysql `connectionString` (use `url`) and `user` (use `username`), and mongo `uri` (use `url`) and `user` (use `username`). #4410 made the authoring gate reject each with a rename hint, but a runtime datasource persisted in `sys_metadata` before the gate kept working only because the factory read leniently — and deleting that tolerance without a conversion would have silently moved data (a stored sqlite `file:` row falls back to `:memory:`). The `datasource-config-driver-key-aliases` conversion rewrites the stored shape to the canonical keys at every rehydration seam, the factory now reads exactly one spelling per key, and the four `??` chains are deleted. Driver-aware by construction: `database` renames only under sqlite, where it aliased the file path — for every other driver it is a canonical key and is untouched. Retired from the load path not for lying but because the authoring gate already rejects the spellings loudly; the chain and the stored-row replay are the seams that accept them.

The `script` flow node converges on its one real path (#4343). It had four ways to name what it ran and only one of them ran anything: `config.actionType: 'email' | 'slack'` were logger-backed stubs that wrote a line, reported success and delivered nothing under any configuration — with `config.template` / `.recipients` / `.variables` feeding a message no channel ever sent; inline `config.script` was recognized and never executed (the built-in runtime has no server-side JS sandbox), so the node warned and no-op'd; and every other `actionType` value was shorthand for a registered-function name, a second spelling of `config.function`. All five keys are retired and `function` becomes required, which is also what finally made the contract PARSEABLE: while the legal key set depended on `actionType`, a flat parse would either reject valid shapes or wave everything through, so `script` (with `subflow`) now runs through the same execute-time contract parse #4277 gave the flat builtins. A shorthand `actionType` CONVERTS into `function` — that is what it meant — unless `function` is already set, in which case it was dead metadata the executor never reached. The other four are dropped outright: nothing read them, so there is no value to preserve, and rebuilding the intent is an authoring decision the tombstones prescribe per branch (a `notify` node for mail — it delivers through the messaging service, the in-app inbox by default and real email once `@objectstack/plugin-email` is installed; a `connector_action` with the Slack connector, or an `http` node posting to a webhook, for Slack; a registered function for an inline body). Retired from the load path for the same reason as the rest: absorbing `actionType: 'email'` silently would let an author keep believing the flow sends mail.

### Mechanical (applied for you)
Expand Down Expand Up @@ -190,6 +192,7 @@ The `script` flow node converges on its one real path (#4343). It had four ways
| `datasource-read-replicas-removed` | `datasource.readReplicas` | datasource key 'readReplicas' removed (#4468 — no driver opened a replica connection and no query path splits reads from writes; front replicas behind one endpoint and point `config` at it) | retired — `migrate meta` only |
| `datasource-capabilities-removed` | `datasource.capabilities` | datasource key 'capabilities' removed (#4583 — eleven flags no code read; pushdown comes from the driver's own supports.*, and `readOnly` never made anything read-only) | retired — `migrate meta` only |
| `datasource-inert-blocks-removed` | `datasource.retryPolicy / datasource.healthCheck / datasource.external.label / datasource.external.requirePermission` | datasource keys 'retryPolicy'/'healthCheck' and external 'label'/'requirePermission' removed (#4583 — nothing retried, nothing probed on a schedule, and the federation label/permission were read by nobody) | retired — `migrate meta` only |
| `datasource-config-driver-key-aliases` | `datasource.config` | datasource config keys → canonical per driver: sqlite 'file'/'database' → 'filename', postgres/mysql 'connectionString' → 'url' and 'user' → 'username', mongo 'uri' → 'url' and 'user' → 'username' (#4456 — driver-factory `??` fallback graduation) | retired — `migrate meta` only |
| `flow-node-script-branch-keys-removed` | `flow.node.script.config.actionType / flow.node.script.config.template / flow.node.script.config.recipients / flow.node.script.config.variables / flow.node.script.config.script` | script flow-node config keys 'actionType' (→ 'function' when it was shorthand for one; otherwise removed — 'email'/'slack' were logger-backed stubs that delivered nothing), plus 'template' / 'recipients' / 'variables' (fed those stubs) and 'script' (inline JS the runtime never executed) (#4343) | retired — `migrate meta` only |

### Semantic (delegated to you, with acceptance criteria)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import { describe, it, expect } from 'vitest';
import type { IDatasourceAdminService, IDatasourceDriverFactory } from '../contracts/index.js';
import type { DatasourceAdminService } from '../datasource-admin-service.js';
import {
DatasourceAdminServicePlugin,
type DatasourceAdminServicePluginOptions,
Expand Down Expand Up @@ -285,6 +286,48 @@ describe('DatasourceAdminServicePlugin: runtime datasource durability', () => {
expect(after.find((d) => d.name === 'demo_ext')?.origin).toBe('runtime');
});

// #4456 — this restore path is a stored-row rehydration seam (ADR-0087 D2
// addendum, #3903): it reads sys_metadata directly, so it must replay the
// conversion chain itself. A row persisted before the #4410 config gate may
// carry the legacy spellings the factory's deleted `??` fallbacks used to
// tolerate; without the replay, a sqlite `file:` row would silently fall
// back to `:memory:` — the data-loss shape the conversion exists to prevent.
it('restores a pre-#4410 row with legacy config keys CANONICAL (conversion chain replayed)', async () => {
const data = fakeSysMetadataEngine();
const now = new Date().toISOString();
for (const [name, driver, config] of [
['legacy_sqlite', 'sqlite', { file: '/tmp/legacy.db' }],
['legacy_pg', 'postgres', { connectionString: 'postgresql://db.internal/analytics', user: 'analyst' }],
['legacy_mongo', 'mongo', { uri: 'mongodb://mongo.internal:27017/events' }],
] as const) {
data.rows.push({
id: `meta_${name}`,
name,
type: 'datasource',
scope: 'platform',
metadata: JSON.stringify({ name, driver, config, origin: 'runtime' }),
state: 'active',
version: 1,
created_at: now,
updated_at: now,
});
}

const b = await boot({ services: { data } });
await b.plugin.start(b.ctx);
// The list DTO is a summary; `getDatasource` (concrete service) is the
// config-bearing read the admin routes serve.
const svc = b.service as unknown as DatasourceAdminService;
expect((await svc.getDatasource('legacy_sqlite'))?.config).toEqual({ filename: '/tmp/legacy.db' });
expect((await svc.getDatasource('legacy_pg'))?.config).toEqual({
url: 'postgresql://db.internal/analytics',
username: 'analyst',
});
expect((await svc.getDatasource('legacy_mongo'))?.config).toEqual({
url: 'mongodb://mongo.internal:27017/events',
});
});

it('removes the durable sys_metadata row when a datasource is deleted', async () => {
const data = fakeSysMetadataEngine();
const b = await boot({ services: { data } });
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -263,3 +263,94 @@ describe('createDefaultDatasourceDriverFactory — declared keys reach the drive
try { await handle.disconnect?.(); } catch { /* pool never opened */ }
});
});

// #4456 — the four undeclared read-side `??` fallbacks are DELETED. The factory
// reads exactly the canonical key of each driver's config contract; the legacy
// spellings a pre-#4410 stored record may carry are rewritten to canonical at
// every rehydration seam by the ADR-0087 conversion
// `datasource-config-driver-key-aliases`, so they must never reach this code —
// and when one does anyway, it is ignored rather than quietly honoured.
describe('createDefaultDatasourceDriverFactory — legacy config spellings are no longer read (#4456)', () => {
function knexConfigOf(driver: any): any {
return driver?.config ?? driver?.knexConfig ?? driver?.options ?? {};
}

async function pgConnection(config: Record<string, unknown>): Promise<any> {
const handle: any = await factory().create({ driver: 'postgres', config });
try { return knexConfigOf(handle.driver ?? handle).connection; }
finally { try { await handle.disconnect?.(); } catch { /* pool never opened */ } }
}

it('pg: `connectionString` no longer selects the DSN path — discrete fields are used instead', async () => {
const conn = await pgConnection({
connectionString: 'postgresql://legacy@db.internal/analytics',
host: 'db.internal',
database: 'analytics',
});
expect(conn.connectionString).toBeUndefined();
expect(conn).toMatchObject({ host: 'db.internal', database: 'analytics' });
});

it('pg: `user` no longer reaches the client — only the canonical `username` does', async () => {
const legacyOnly = await pgConnection({ host: 'h', database: 'd', user: 'legacy' });
expect(legacyOnly.user).toBeUndefined();
const both = await pgConnection({ host: 'h', database: 'd', user: 'legacy', username: 'svc' });
expect(both.user).toBe('svc');
});

it('mysql: `connectionString`/`user` are ignored the same way', async () => {
const handle: any = await factory().create({
driver: 'mysql',
config: { connectionString: 'mysql://legacy@db/orders', host: 'db', database: 'orders', user: 'legacy' },
});
const conn = knexConfigOf(handle.driver ?? handle).connection;
// Not the DSN string passthrough — the discrete-field object, with no user.
expect(typeof conn).toBe('object');
expect(conn).toMatchObject({ host: 'db', database: 'orders' });
expect(conn.user).toBeUndefined();
try { await handle.disconnect?.(); } catch { /* pool never opened */ }
});

it('sqlite-wasm: `file`/`database` no longer name the database — the driver builds `:memory:`', async () => {
const dir = mkdtempSync(join(tmpdir(), 'os-4456-'));
try {
const legacyFile = join(dir, 'legacy.db');
const handle: any = await factory().create({
driver: 'sqlite-wasm',
config: { file: legacyFile, database: legacyFile },
});
const driver = handle.driver ?? handle;
await driver.connect();
try {
await driver.syncSchema('note', { name: 'note', fields: { id: { type: 'text' } } });
await driver.create('note', { id: 'n1' });
} finally {
try { await driver.disconnect(); } catch { /* noop */ }
}
// An ephemeral `:memory:` database writes nothing at the legacy path.
expect(existsSync(legacyFile)).toBe(false);
} finally {
try { rmSync(dir, { recursive: true, force: true }); } catch { /* noop */ }
}
}, 30_000);

it('mongo: `uri`/`user` are ignored — the URL is composed from canonical keys only', async () => {
const handle: any = await factory().create({
driver: 'mongo',
config: { uri: 'mongodb://legacy.internal:27017/legacy', database: 'events', user: 'legacy' },
});
const driver: any = handle.driver ?? handle;
// No canonical `url`/`host`/`username` → composed from defaults + `database`,
// with no auth part; the legacy `uri` never passes through.
expect(driver.config.url).toBe('mongodb://localhost:27017/events');
});

it('mongo: the canonical spellings still compose the URL (control)', async () => {
const handle: any = await factory().create({
driver: 'mongo',
config: { host: 'mongo.internal', port: 27017, database: 'events', username: 'svc', password: 'pw' },
});
const driver: any = handle.driver ?? handle;
expect(driver.config.url).toBe('mongodb://svc:pw@mongo.internal:27017/events');
});
});
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.

import type { Plugin, PluginContext } from '@objectstack/core';
import { applyConversionsToStoredItem } from '@objectstack/spec';
import { registerMetadataTypeActions } from '@objectstack/spec/kernel';
import type {
IDatasourceDriverFactory,
Expand Down Expand Up @@ -107,7 +108,15 @@ async function loadDatasourceRows(engine: DataEngineLike | undefined): Promise<A
for (const r of rows ?? []) {
const raw = (r as { metadata?: unknown }).metadata;
try {
out.push(typeof raw === 'string' ? JSON.parse(raw) : (raw as Record<string, unknown>));
const parsed = typeof raw === 'string' ? JSON.parse(raw) : (raw as Record<string, unknown>);
// This is a stored-row rehydration seam (ADR-0087 D2 addendum, #3903):
// rows written under a past protocol replay the FULL conversion chain —
// e.g. a pre-#4410 sqlite record whose config still says `file:` is
// served with the canonical `filename`, which is the only spelling the
// driver factory reads since #4456. The direct sys_metadata read here
// bypasses the metadata service's own converting loaders, so the pass
// must happen locally.
out.push(applyConversionsToStoredItem(DS_META_TYPE, parsed));
} catch {
/* skip corrupt row */
}
Expand Down
Loading
Loading