Skip to content

fix(drizzle-kit): limit prepared Postgres query concurrency#24

Merged
joe-thom merged 2 commits intomainfrom
05-05-fix_drizzle-kit_limit_prepared_postgres_query_concurrency
May 5, 2026
Merged

fix(drizzle-kit): limit prepared Postgres query concurrency#24
joe-thom merged 2 commits intomainfrom
05-05-fix_drizzle-kit_limit_prepared_postgres_query_concurrency

Conversation

@joe-thom
Copy link
Copy Markdown

@joe-thom joe-thom commented May 5, 2026

Why

Replit's deploy database-preparation flow uses the drizzle-kit/api helpers to introspect user databases. Datadog shows the current failure mode is not TCP connection startup; it is prepareDatabaseMigrations timing out during schema introspection with pg.Pool.waitingCount in the hundreds/thousands.

The root cause is that drizzle-kit introspection fans out one async task per table/view. Each task runs multiple db.query() calls, so large schemas can enqueue thousands of pg-pool acquires behind a small migration pool.

What changed

Added an optional queryConcurrency setting to preparePgDB().

When set, both query() and proxy() calls returned by preparePgDB() go through a small shared concurrency limiter before calling pool.query(...). Existing callers that do not pass the option keep the previous unbounded behavior.

This lets repl-it-web call:

preparePgDB(pool, { queryConcurrency: 4 })

so drizzle-kit introspection does not dump thousands of queued acquires into pg.Pool.

Test plan

  • pnpm --filter @drizzle-team/drizzle-kit exec vitest run tests/api.test.ts passed
  • pnpm --filter @drizzle-team/drizzle-kit run tsc passed
  • pnpm --filter @drizzle-team/drizzle-kit run build passed
  • pnpm lint passed
  • I also accidentally started the broader drizzle-kit test suite; it ran for 3 minutes and many suites passed, then my tool timeout killed it before completion. No failures were observed before timeout.

Revertibility

Fully revertible. This is an additive API option and preserves existing behavior when queryConcurrency is omitted.

Follow-up

After this package is published, repl-it-web should bump @drizzle-team/drizzle-kit and pass queryConcurrency: 4 from the publish database-preparation flow.

Copy link
Copy Markdown
Author

joe-thom commented May 5, 2026

@joe-thom joe-thom changed the title fix(drizzle-kit): limit prepared postgres query concurrency fix(drizzle-kit): limit prepared Postgres query concurrency May 5, 2026
@joe-thom joe-thom marked this pull request as ready for review May 5, 2026 12:02
Comment thread drizzle-kit/src/api.ts Outdated
@joe-thom joe-thom merged commit 14a582f into main May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants