Skip to content

migrate: drop-column is emitted before drop-fk, so dropping a referenced column always fails #255

Description

@dmealing

Affected port(s): TypeScript (emitter defect; SQLite emitter carries the same table)
Package + version: @metaobjectsdev/cli + @metaobjectsdev/migrate-ts 0.20.5

What happened

Adopting an existing Postgres database — meta migrate baseline --from-db, then
meta migrate --slug reconcile --allow drop-column,drop-fk,drop-table,… --apply
aborts on the first dropped column that a foreign key still references:

meta: migrate: apply failed: cannot drop column user_id of table user_profiles
      because other objects depend on it

The generated migration is un-appliable. No flag or metadata change works around it:
drop-column and drop-fk are both already permitted via --allow; the problem is
the order the statements are emitted in.

The apply is transactional, so the database rolls back cleanly — no partial state.
That is the only reason this is a blocker rather than a corruption bug.

What you expected

meta migrate --apply emits ALTER TABLE … DROP CONSTRAINT for the dependent
foreign keys before ALTER TABLE … DROP COLUMN removes the column they reference,
so the migration applies.

Root cause

server/typescript/packages/migrate-ts/src/emit/postgres.ts, STAGE_ORDER
(statements are sorted low → high, ~line 23):

  "add-column": 2, "drop-column": 2,        // <-- drop-column runs at stage 2
  
  "add-fk": 5, "drop-fk": 5,                // <-- but drop-fk not until stage 5
  "add-check": 5, "drop-check": 5,
  "drop-table": 6,

drop-column (2) is emitted before drop-fk (5), so the column is still
referenced when ALTER TABLE … DROP COLUMN runs and Postgres refuses.

The header comment shows the dependency was reasoned about for tables but never
extended to columns:

drop-view + drop-fk run BEFORE drop-table so a view that depends on a soon-to-be-dropped table is removed first.

That holds — drop-table is stage 6, after drop-fk at 5. The identical argument
applies to drop-column and was not made.

The underlying modelling error: constraint drops and constraint adds share one
stage
, but they have opposite ordering requirements.

  • add-fk / add-check must run after column changes (the columns must exist).
  • drop-fk / drop-check must run before column changes (the constraint must be
    gone before its column can be dropped).

One stage cannot satisfy both.

Suggested fix

Split the constraint stages so drops precede column mutation and adds follow it:

  "drop-view": 0,
  "drop-fk": 1, "drop-check": 1,            // NEW: before any column mutation
  "create-table": 1,
  "add-column": 2, "drop-column": 2,
  
  "add-fk": 5, "add-check": 5,              // adds stay late
  "drop-table": 6,

Renumber to taste; the invariant is drop-fk < drop-column < add-fk, with drop-fk
still before drop-table (any stage < 6).

src/emit/sqlite.ts carries the identical table ("add-column": 2, "drop-column": 2
"add-fk": 5, "drop-fk": 5) and needs the same change. Not observed failing there
only because SQLite's recreate-and-copy path masks it for some shapes.

Reproduction

Any schema where a column targeted for drop is referenced by a foreign key:

parent(id PK, legacy_key)                            -- metadata drops parent.legacy_key
child(…, FK child.parent_key -> parent.legacy_key)   -- metadata drops this FK too
meta migrate --from-db --db postgresql://… --dialect postgres \
  --slug repro --allow drop-column,drop-fk --apply

The DROP COLUMN is emitted first and Postgres rejects it.

Environment

Linux, Node 24, Postgres 16, npm. Reproduced against a restored copy of a real
database (~200 pending changes) rather than a synthetic fixture.

Note for whoever picks this up

In the reconcile that surfaced this there were also 49 add-fk changes against
tables with existing rows. Those never executed — the run aborted at the column drop
— so whether they apply cleanly against real data is still unverified. Expect a
possible second failure class behind this one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions