test(migrate-ts): green the Postgres integration release gate by dmealing · Pull Request #280 · metaobjectsdev/metaobjects

dmealing · 2026-08-09T01:22:27Z

Intent

Green the migrate-ts-pg release gate, which has been failing on EVERY release tag going back to at least v0.20.0 — the same 6 tests on v0.21.0 and v0.21.1, verified byte-identical between those two runs.

WHY IT WENT UNNOTICED (structural, not carelessness). Three compounding blind spots: the migrate-ts-pg job runs ONLY on v* tag push or manual dispatch, never on PR or merge — so the red arrives AFTER publishing, when the irreversible step is already done; the same tests describe.skip SILENTLY when MIGRATE_TS_PG_URL is unset, so local runs and PR runs both look green; and nobody re-checks a tag run once the packages are out. A release gate that is permanently red provides zero signal, and that is precisely how the #279 legacy-Postgres-serial-PK adoption bug (which would drop a live table's PK default and break every insert) reached a release without being caught.

BOTH ROOT CAUSES ARE TEST-SIDE. There is no product defect here, and no product file is touched.

(1) SOURCELESS FIXTURES — 4 of the 6 failures. postgres-lenient-inet.test.ts and pg-adversarial-fixes.test.ts each declared an object.entity with NO source.rdb child. Persistability derives from a declared or inherited source.* child (#249, "derive persistability from source presence, not object subtype"), so a sourceless entity is not persistable: no CREATE TABLE is emitted, the table never exists, the first assertion fails on a missing column, and the remaining tests in the file cascade as relation "endpoints" does not exist / relation "readings" does not exist. This was PROVEN, not inferred — adding source.rdb alone, with no other change, took each file from all-fail to all-pass, and the change was reverted and re-applied deliberately to confirm.

(2) STALE CHECK-EXPRESSION SPACING — the other 2. lifecycle-pg.test.ts still expected status in 'open', 'closed' with spaces after the commas, but its own checkExprs helper runs the product's normalizeCheckExpr, which deliberately collapses comma spacing outside string literals (from the 0.20.2 CHECK-reconciliation work). The normalizer is correct; the expectations predate it and were never updated, because this lane never ran anywhere they would be seen.

VERIFICATION. Against a real Postgres 16 in a throwaway container: the full migrate-ts integration suite goes from 124 pass / 6 fail to 130 pass / 0 fail. The no-PG path still skips cleanly (688 pass / 19 skip / 0 fail), so PR runs and local runs are unaffected by these changes. Each failing file was also run in ISOLATION first, which falsified an earlier hypothesis that the failures were cross-file interference from shared-database test ordering — they fail alone too, so the cause is deterministic and per-fixture.

ONE CORRECTION TO AN EARLIER CLAIM, recorded deliberately: I initially reported the lane went red at v0.20.11 and suspected the #249 persistability change caused it. Checking further back showed the lane was ALREADY red at v0.20.0, so the redness has a longer, multi-cause history and should not be pinned on a single commit. What is proven is the current six and their two causes.

DELIBERATELY NOT CHANGED: the lane's tag-only trigger. Making migrate-ts-pg run on merge (or on PRs touching migrate-ts) is a workflow policy decision for the maintainer, not something a test fix should smuggle in. But until that changes, this lane can silently rot again — the structural half of the problem remains open by design, not by oversight.

Scope: three test files under server/typescript/packages/migrate-ts/test/integration/. No product code, no metadata vocabulary, no workflow files.

What Changed

Added a source.rdb child (@table) to the sourceless Endpoint and Reading entity fixtures in postgres-lenient-inet.test.ts and pg-adversarial-fixes.test.ts — without a declared/inherited source these entities are non-persistable (fix: derive persistability from source presence, not object subtype #249), so no CREATE TABLE was emitted and every assertion in the file cascaded into relation ... does not exist.
Updated the CHECK-expression spacing expectations in lifecycle-pg.test.ts (status in 'open', 'closed' → status in 'open','closed') to match the product's normalizeCheckExpr, which deliberately collapses comma spacing outside string literals; the normalizer is correct, the stale expectations predated it.

No product code is touched — both root causes are test-side. The full migrate-ts suite against real Postgres goes from 124 pass / 6 fail to 130 pass / 0 fail; the no-PG path still skips cleanly (688 pass / 19 skip / 0 fail), so PR and local runs are unaffected.

Risk Assessment

✅ Low: A well-bounded, test-only fix across three integration fixtures; both root causes are verified against product source (persistability derives from source presence per expected-schema.ts:146-156; normalizeCheckExpr collapses comma spacing outside quotes per check-expr-compare.ts:46,50-82), no product/workflow/metadata code is changed, and a diligence scan found no other latent instances of either pattern in the PG lane.

Testing

Reproduced the migrate-ts-pg CI gate exactly (postgres:16 sidecar, same credentials, the gate's own bun test command). With PG set the full package is 708 pass / 0 fail (was 6 fail); without PG it is 688 pass / 19 skip / 0 fail, matching the intent's claim that PR/local runs are unaffected. A controlled before/after (revert the 3 files to base → 6 failures with the two documented causes: sourceless endpoints/readings fixtures and stale CHECK-expr comma spacing; restore → 8 pass / 0 fail) proves the fix is deterministic and addresses both root causes. Only test files changed; no product code touched. Transient container removed and worktree left clean at the target commit.

Evidence: Full migrate-ts suite vs real Postgres (gate green)

bun test v1.3.14 (0d9b296a) 708 pass 0 fail 1498 expect() calls Ran 708 tests across 119 files. [5.37s]

bun test v1.3.14 (0d9b296a)

 708 pass
 0 fail
 1498 expect() calls
Ran 708 tests across 119 files. [5.37s]

Evidence: Full migrate-ts suite, no PG (skip-cleanly check)

bun test v1.3.14 (0d9b296a) 688 pass 19 skip 0 fail 1345 expect() calls Ran 707 tests across 119 files. [3.66s]

bun test v1.3.14 (0d9b296a)

 688 pass
 19 skip
 0 fail
 1345 expect() calls
Ran 707 tests across 119 files. [3.66s]

Evidence: Before-state: 3 base-version files vs real PG (6 failures reproduced)

(base commit, 3 files vs PG) 2 pass / 6 fail — root causes reproduced: • postgres-lenient-inet: error: relation "endpoints" does not exist (×2, sourceless fixture) • pg-adversarial-fixes: error: relation "readings" does not exist (×2, sourceless fixture) • lifecycle-pg: expect(checkExprs).toEqual/toContain failed on 'status in 'open', 'closed'' vs collapsed 'status in 'open','closed'' (×2, stale CHECK-expr spacing) After restoring target versions: 8 pass / 0 fail.

bun test v1.3.14 (0d9b296a)

test/integration/postgres-lenient-inet.test.ts:
87 |     const colTypes = await sql<{ column_name: string; data_type: string }>`
88 |       SELECT column_name, data_type FROM information_schema.columns
89 |       WHERE table_name = 'endpoints' ORDER BY column_name
90 |     `.execute(kysely);
91 |     const byName = new Map(colTypes.rows.map((r) => [r.column_name, r.data_type]));
92 |     expect(byName.get("strict_ip")).toBe("inet");     // native
                                         ^
error: expect(received).toBe(expected)

Expected: "inet"
Received: undefined

      at <anonymous> (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/server/typescript/packages/migrate-ts/test/integration/postgres-lenient-inet.test.ts:92:37)
(fail) #234 — @lenient field.inet PG idempotence + value semantics > IDEMPOTENCE: strict inet stays `inet`, lenient inet is `text` — apply then re-diff is [] [14.35ms]
689 |         result = new this._Promise((resolve, reject) => {
690 |           query.callback = (err, res) => (err ? reject(err) : resolve(res))
691 |         }).catch((err) => {
692 |           // replace the stack trace that leads to `TCP.onStreamRead` with one that leads back to the
693 |           // application that created the query
694 |           Error.captureStackTrace(err)
                      ^
error: relation "endpoints" does not exist
     length: 108,
   severity: "ERROR",
     detail: undefined,
       hint: undefined,
   position: "13",
 internalPosition: undefined,
 internalQuery: undefined,
      where: undefined,
     schema: undefined,
      table: undefined,
   dataType: undefined,
 constraint: undefined,
       file: "parse_relation.c",
    routine: "parserOpenTable",
       code: "42P01"

      at /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/pg@8.23.0+00a0136bc273dfed/node_modules/pg/lib/client.js:694:17
      at processTicksAndRejections (native:7:39)
      at executeQuery (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/dialect/postgres/postgres-driver.js:109:23)
      at /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:48:49
      at provideConnection (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/driver/default-connection-provider.js:12:9)
      at executeQuery (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:46:29)
      at processTicksAndRejections (native:7:39)
(fail) #234 — @lenient field.inet PG idempotence + value semantics > VALUE SEMANTICS: a not-strictly-valid value stores in the lenient `text` column and is rejected by the strict `inet` column [0.42ms]

test/integration/pg-adversarial-fixes.test.ts:
689 |         result = new this._Promise((resolve, reject) => {
690 |           query.callback = (err, res) => (err ? reject(err) : resolve(res))
691 |         }).catch((err) => {
692 |           // replace the stack trace that leads to `TCP.onStreamRead` with one that leads back to the
693 |           // application that created the query
694 |           Error.captureStackTrace(err)
                      ^
error: relation "readings" does not exist
     length: 107,
   severity: "ERROR",
     detail: undefined,
       hint: undefined,
   position: "13",
 internalPosition: undefined,
 internalQuery: undefined,
      where: undefined,
     schema: undefined,
      table: undefined,
   dataType: undefined,
 constraint: undefined,
       file: "parse_relation.c",
    routine: "parserOpenTable",
       code: "42P01"

      at /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/pg@8.23.0+00a0136bc273dfed/node_modules/pg/lib/client.js:694:17
      at processTicksAndRejections (native:7:39)
      at executeQuery (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/dialect/postgres/postgres-driver.js:109:23)
      at /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:48:49
      at provideConnection (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/driver/default-connection-provider.js:12:9)
      at executeQuery (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:46:29)
      at processTicksAndRejections (native:7:39)
(fail) PG adversarial fixes — real-engine value semantics + idempotence > @isArray scalars: array INSERT works, engine stores real arrays, re-diff empty [7.72ms]
689 |         result = new this._Promise((resolve, reject) => {
690 |           query.callback = (err, res) => (err ? reject(err) : resolve(res))
691 |         }).catch((err) => {
692 |           // replace the stack trace that leads to `TCP.onStreamRead` with one that leads back to the
693 |           // application that created the query
694 |           Error.captureStackTrace(err)
                      ^
error: relation "readings" does not exist
     length: 107,
   severity: "ERROR",
     detail: undefined,
       hint: undefined,
   position: "13",
 internalPosition: undefined,
 internalQuery: undefined,
      where: undefined,
     schema: undefined,
      table: undefined,
   dataType: undefined,
 constraint: undefined,
       file: "parse_relation.c",
    routine: "parserOpenTable",
       code: "42P01"

      at /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/pg@8.23.0+00a0136bc273dfed/node_modules/pg/lib/client.js:694:17
      at processTicksAndRejections (native:7:39)
      at executeQuery (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/dialect/postgres/postgres-driver.js:109:23)
      at /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:48:49
      at provideConnection (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/driver/default-connection-provider.js:12:9)
      at executeQuery (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/node_modules/.bun/kysely@0.27.6/node_modules/kysely/dist/esm/query-executor/query-executor-base.js:46:29)
      at processTicksAndRejections (native:7:39)
(fail) PG adversarial fixes — real-engine value semantics + idempotence > quote-bearing @default: engine stores the un-escaped value; no bogus SET DEFAULT [4.07ms]

test/integration/lifecycle-pg.test.ts:
156 |     const orders1 = live[0]!;
157 |     expect(colNames(orders1)).toEqual(["email", "id", "qty", "status"]);
158 |     expect(orders1.primaryKey).toEqual(["id"]);
159 |     expect(orders1.indexes.some((i) => i.unique && i.columns.includes("email"))).toBe(true);
160 |     // enum + numeric checks present
161 |     expect(checkExprs(orders1)).toEqual(
                                      ^
error: expect(received).toEqual(expected)

@@ -2,3 +2,3 @@
    "qty >= 1",
-   "status in 'open', 'closed'",
+   "status in 'open','closed'",
  ]

- Expected  - 1
+ Received  + 1

      at <anonymous> (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/server/typescript/packages/migrate-ts/test/integration/lifecycle-pg.test.ts:161:33)
(fail) migrate-ts lifecycle against real Postgres > full lifecycle: create → apply → evolve → apply → rollback → idempotent → gated [36.19ms]
324 |       const posts = gf.find((t) => t.name === "gf_posts")!;
325 |       const comments = gf.find((t) => t.name === "gf_comments")!;
326 |       expect(posts.foreignKeys.some((f) => f.refTable === "gf_users")).toBe(true);
327 |       expect(comments.foreignKeys.map((f) => f.refTable).sort()).toEqual(["gf_posts", "gf_users"]);
328 |       // enum + numeric checks + unique indexes survived the round-trip
329 |       expect(checkExprs(gf.find((t) => t.name === "gf_users")!)).toContain("role in 'admin', 'editor', 'viewer'");
                                                                       ^
error: expect(received).toContain(expected)

Expected to contain: "role in 'admin', 'editor', 'viewer'"
Received: [ "role in 'admin','editor','viewer'" ]

      at <anonymous> (/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZJ13S99V28K69Y4ZMFWG3J6/server/typescript/packages/migrate-ts/test/integration/lifecycle-pg.test.ts:329:66)
(fail) migrate-ts lifecycle against real Postgres > greenfield first deploy: a 4-entity app schema created from an empty database [69.84ms]

 2 pass
 6 fail
 37 expect() calls
Ran 8 tests across 3 files. [294.00ms]

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

git diff 35b0250c0..396b860d5 (scope: 3 test files, 6 line-changes, no product code)
MIGRATE_TS_PG_URL=postgres://migrate:migrate@localhost:5432/migrate_test bun test test/integration/lifecycle-pg.test.ts test/integration/pg-adversarial-fixes.test.ts test/integration/postgres-lenient-inet.test.ts (3 changed files in isolation vs real PG → 8 pass / 0 fail)
MIGRATE_TS_PG_URL=... bun test (full migrate-ts package vs real PG, the literal gate command → 708 pass / 0 fail across 119 files)
env -u MIGRATE_TS_PG_URL bun test (full package, no PG → 688 pass / 19 skip / 0 fail)
git checkout 35b0250c0 -- <3 files> then bun test vs PG (before-state reproduced the 6 failures: endpoints/readings 'does not exist' + CHECK-expr spacing)
git checkout HEAD -- <3 files> then bun test vs PG (after-state restored: 8 pass / 0 fail)

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

The `migrate-ts-pg` job in integration-tests.yml has failed on EVERY release tag back to at least v0.20.0 -- the same 6 tests on v0.21.0 and v0.21.1. It went unnoticed structurally, not carelessly: the lane runs ONLY on `v*` tag push, so the red arrives AFTER publishing, and the same tests `describe.skip` silently without MIGRATE_TS_PG_URL, so local and PR runs look green. A release gate that is permanently red provides no signal, which is how the #279 serial-PK adoption bug reached a release. Both root causes are test-side; no product defect. 1. Sourceless fixtures (4 of 6 failures). `postgres-lenient-inet` and `pg-adversarial-fixes` declared an `object.entity` with NO `source.rdb` child. Persistability derives from a declared/inherited `source.*` (#249), so those entities are non-persistable: no CREATE TABLE is emitted, the table never exists, and the assertions fail -- then cascade as `relation "endpoints"/"readings" does not exist`. Proven, not inferred: adding `source.rdb` alone takes each file from all-fail to all-pass. 2. Stale CHECK-expression spacing (2 of 6). The expectations still carried `'open', 'closed'` with spaces after the commas, but `checkExprs` runs `normalizeCheckExpr`, which deliberately collapses comma spacing outside literals. The normalizer is right; the expectations predate it. Verified against a real Postgres 16: the full integration suite goes 124 pass / 6 fail -> 130 pass / 0 fail. The no-PG path still skips cleanly (688 pass / 19 skip / 0 fail), so PR runs are unaffected. Deliberately NOT changed: the lane's tag-only trigger. Making it run on merge is a workflow policy decision for the maintainer, not a test fix -- but until it does, this lane can silently rot again. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TqsuDye2SfXGf43vuoD3n

dmealing merged commit 0c8e11b into main Aug 9, 2026
1 check passed

dmealing deleted the fix/migrate-ts-pg-lane-green branch August 9, 2026 01:23

Navigation Menu

Uh oh!

test(migrate-ts): green the Postgres integration release gate - #280

test(migrate-ts): green the Postgres integration release gate#280
dmealing merged 1 commit into
mainfrom
fix/migrate-ts-pg-lane-green

dmealing commented Aug 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

dmealing commented Aug 9, 2026

Intent

What Changed

Risk Assessment

Testing

Pipeline

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant