fix: use default pool in prod, not the test SQL sandbox - #88
Merged
Conversation
prod.exs configured Ethui.Repo with pool: Ecto.Adapters.SQL.Sandbox — a verbatim copy of config/test.exs. The sandbox pool is DBConnection.Ownership; in its default :auto mode it assigns a connection to each process on first use and only releases it when that process terminates (ownership manager handle_info(:DOWN), db_connection manager.ex:203). It does NOT check the connection back in after each query, unlike DBConnection.ConnectionPool. So every long-lived process that touches the DB (telemetry poller, per-stack servers, ...) permanently holds one of the pool_size connections. After ~10 such processes the pool is exhausted for good: even a single manual Repo query from a fresh iex session times out with a checkout error, which is what we observed in production. Removing the pool: line falls back to the default DBConnection.ConnectionPool, which returns connections after each operation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The prod.exs pool_size (schedulers*2) was dead — runtime.exs always overrides it (prod ran pool_size: 10). Drop it so runtime.exs is the single source; keep default_transaction_mode: :immediate (recommended for SQLite). Verified with a reproduction: 2 long-lived processes that each run one query exhaust the sandbox pool permanently (connections held per-process until the process dies), while the default DBConnection pool returns them after each query. Matches the Phoenix --database sqlite3 template, which uses the default pool in prod and the sandbox only in test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DavideSilva
approved these changes
Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause (reproduced + confirmed on the running prod node)
config/prod.exssetEthui.Repotopool: Ecto.Adapters.SQL.Sandbox— a verbatim copy ofconfig/test.exs.The sandbox pool is
DBConnection.Ownership. In its default:automode it assigns a connection to a process on first DB use and only releases it when that process terminates (manager.ex:203+ the:DOWNhandler). Unlike the defaultDBConnection.ConnectionPool, it does not check the connection back in after each query.So every long-lived process that queries the DB (the telemetry poller running
publish_active_stacks_count, per-stack servers, …) permanently holds one of thepool_sizeconnections. After ~pool_sizesuch processes the pool is exhausted for good.Reproduced locally
2 long-lived processes that each run ONE query then stay alive,
pool_size: 2:Confirmed on prod
A single
Ethui.Accounts.get_api_key_by_token(token)from a freshiex— zero request load — times out withcould not checkout. The token is valid and cached (ETS returns the%ApiKey{}), so cache-served requests still 200'd, masking the dead pool.Matches the Phoenix template
mix phx.new --database sqlite3putspool: Sandboxonly intest.exs; prod uses the default pool. This change makes prod match that.Change
pool: Ecto.Adapters.SQL.Sandbox→ prod uses the defaultDBConnection.ConnectionPool, which returns connections after each operation.pool_sizeinprod.exs(it wasschedulers*2butruntime.exsalways overrode it toPOOL_SIZE/10).runtime.exsis now the single source. Keepdefault_transaction_mode: :immediate(recommended for SQLite).Verify after deploy
SQLite pool_size note (not changed here)
SQLite is single-writer; large pools can cause
database is lockedunder heavy concurrent writes. Current prodpool_sizeis 10 (runtime, env-tunable viaPOOL_SIZE); the sqlite template default is 5. Left at 10 — light write load, and WAL +busy_timeout: 2000+:immediatemitigate. LowerPOOL_SIZEifdatabase is lockedshows up.Note
This is the actual fix for the checkout-timeout 500s. The earlier api-key ETS cache (already merged) is a valid optimization but only masked the symptom.
🤖 Generated with Claude Code