Skip to content

WPB-24882 improve connection pool management - #5375

Merged
battermann merged 1 commit into
developfrom
WPB-26489-upgrade-to-latest-arbiter-version
Jul 28, 2026
Merged

WPB-24882 improve connection pool management#5375
battermann merged 1 commit into
developfrom
WPB-26489-upgrade-to-latest-arbiter-version

Conversation

@battermann

@battermann battermann commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

https://wearezeta.atlassian.net/browse/WPB-27400

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

@battermann
battermann marked this pull request as ready for review July 27, 2026 15:22
@battermann
battermann requested review from a team as code owners July 27, 2026 15:22
@battermann
battermann requested a review from Copilot July 27, 2026 15:22
@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Jul 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR (WPB-26489) updates Arbiter integration to improve PostgreSQL connection pool management by ensuring background-worker job execution uses the shared Hasql pool (and avoids dedicating extra long-lived connections), alongside an Arbiter dependency bump and related API adjustments.

Changes:

  • Switch background-worker Arbiter workers to a pool-backed, poll-only execution model (no LISTEN/NOTIFY listener connection; shared Hasql pool for DB work).
  • Move/centralize Arbiter migration connection-string construction into wire-subsystems (mkArbiterConnectionString) and update services to use it.
  • Update Arbiter-related types/APIs (e.g., JobRegistry definition) and align Nix/Cabal deps plus operator documentation/changelog.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated no comments.

Show a summary per file
File Description
services/galley/src/Galley/Run.hs Uses mkArbiterConnectionString for Arbiter migrations; keeps Hasql pool usage unchanged.
services/background-worker/src/Wire/BackgroundWorker/Workers.hs Switches worker pools to transactionalWorkerConfig, removes dedicated Arbiter conn-str usage, and uses pool shutdown helpers.
services/background-worker/src/Wire/BackgroundWorker/Env.hs Builds Arbiter migration connection string via mkArbiterConnectionString.
nix/haskell-pins.nix Adds arbiter-worker-testkit pin entry.
libs/wire-subsystems/wire-subsystems.cabal Adds postgresql-connection-string dependency to support new conn-str builder.
libs/wire-subsystems/src/Wire/JobSubsystem/Migrations.hs Introduces mkArbiterConnectionString and clarifies migration-lock connection lifecycle.
libs/wire-subsystems/src/Wire/JobSubsystem/Interpreter.hs Updates Arbiter API usage for inserting jobs and query execution.
libs/wire-subsystems/src/Wire/JobSubsystem/ArbiterAdapter.hs Adapts to newer Arbiter MonadArbiter API (Query wrapper, placeholder numbering) and disables listener usage.
libs/wire-subsystems/default.nix Adds postgresql-connection-string to Nix derivation inputs.
libs/wire-api/wire-api.cabal Adds arbiter-core dependency for updated registry typing.
libs/wire-api/src/Wire/API/Jobs.hs Updates JobRegistry to use Queue type from arbiter-core.
libs/wire-api/default.nix Adds arbiter-core to Nix derivation inputs.
libs/extended/src/Hasql/Pool/Extended.hs Removes the old connection-string helper (now provided by wire-subsystems).
flake.nix Bumps Arbiter source revision.
flake.lock Updates Arbiter lock metadata to match new revision.
docs/src/developer/reference/config-options.md Updates operator docs to reflect shared-pool concurrency and removal of LISTEN/NOTIFY connection.
changelog.d/0-release-notes/WPB-26489 Updates release notes to match new connection/pool behavior and migration-lock connection lifecycle.

@blackheaven blackheaven left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you have to upgrade?

@battermann

Copy link
Copy Markdown
Contributor Author

Why did you have to upgrade?

Then new version offers better use of postgres connections, keeping configuration tuning and overhead for operators to a minimum.

@battermann
battermann merged commit 8a5e78f into develop Jul 28, 2026
11 checks passed
@battermann
battermann deleted the WPB-26489-upgrade-to-latest-arbiter-version branch July 28, 2026 07:44
@battermann battermann changed the title WPB-26489 improve connection pool management WPB-24882 improve connection pool management Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants