Skip to content

db reset: intermittent "error running container: exit 1" at "Initialising schema" (one-off migration jobs race the running storage service) #6445

Description

@milgron

Bug report

supabase db reset fails intermittently at "Initialising schema..." with error running container: exit 1 (surfaced by the TypeScript CLI as LegacyDbSetupError). Roughly 1 in 6 to 20 resets on a machine that runs the reset in a loop (pre-commit hook: reset, then supabase test db).

Describe the bug

Sequence printed by the CLI on a failing run:

Resetting local database...
Recreating database...
Initialising schema...
{"_tag":"Error","error":{"code":"LegacyDbSetupError","message":"error running container: exit 1"}}

The failure is inside the reset itself, before any user migration or seed runs. pg_prove afterwards reports every test file with "planned N tests but ran 0" because the schema is missing, which is a consequence, not the cause.

What I think is happening

In apps/cli-go/internal/db/start/start.go, initSchema15 runs one-off Docker jobs for realtime, storage (node dist/scripts/migrate-call.js) and auth (gotrue migrate) against the recreated postgres database. At the same time the long-running service containers reconnect to the recreated database, and at least storage-api runs its own migrations on reconnect. Its container log shows, on every reset:

{"type":"migrations","msg":"[Migrations] Running vector_store migrations"}
{"type":"migrations","msg":"[Migrations] Created database storage_vectors"}

So two migrators race on the same schema during "Initialising schema..." and one of them exits non-zero. That matches the timing-dependent nature: with --debug (slower logging) I could not reproduce it in 22 consecutive resets, without it the failure shows up every handful of runs.

To Reproduce

  1. supabase init, supabase start -x vector,logflare,studio,imgproxy,inbucket,mailpit,edge-runtime (storage, auth, realtime, rest, kong, db and pg_meta running).
  2. Add any migration and a seed.sql.
  3. Loop: for i in $(seq 1 30); do supabase db reset || break; supabase test db; done

One of the resets fails with the message above.

Expected behavior

supabase db reset should either pause the running service containers while the one-off migration jobs run, or run the service migrations only once.

System information

  • OS: Linux 6.18 (Manjaro), Docker 29.6.2
  • Version of supabase-js: n/a
  • Version of Node.js: 24.16
  • Version of supabase CLI: 2.116.0 (devDependency)
  • Images: postgres 17.6.1.165, storage-api 1.70.3, realtime v2.129.3
  • config.toml: default from supabase init, [db.seed] sql_paths = ["./seed.sql"]

Workaround

Retrying supabase db reset once succeeds every time. Excluding realtime from the local stack reduces the number of concurrent migrators but storage-api still races.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions