Skip to content

Add member_order (fifo/lifo) + demand-history cull#110

Merged
seriyps merged 1 commit into
epgsql:masterfrom
seriyps:member-order
Jun 2, 2026
Merged

Add member_order (fifo/lifo) + demand-history cull#110
seriyps merged 1 commit into
epgsql:masterfrom
seriyps:member-order

Conversation

@seriyps

@seriyps seriyps commented May 29, 2026

Copy link
Copy Markdown
Member
  • Add member_order => lifo | fifo pool config (lifo is the default and preserves
    all existing behaviour). fifo uses an OTP queue as the free-member structure,
    handing out the longest-idle worker first (round-robin rotation).

  • Replace the per-worker max_age idle-time cull gate with pool-level demand
    history
    : a fixed-capacity queue ring buffer (head = current bucket) tracks peak
    concurrent in-use count over a rolling max_age window. At each cull tick the pool
    sizes to that peak (floored at init_count). This gives LIFO and FIFO identical
    shrinkage behaviour for the same config — FIFO workers never go idle under rotation,
    so the old per-worker check never fired for them.

  • max_age now means "demand memory window" rather than "individual worker idle time".
    Practical behaviour for LIFO pools is essentially unchanged; FIFO pools now cull
    correctly. cull_interval default changed from {1, min} to {15, sec} so the
    defaults satisfy cull_interval ≤ max_age.

  • Fixed-size pools (init_count == max_count) skip demand tracking entirely
    (demand_buf = undefined) — cull never fires for them anyway.

  • Hot upgrade v5 → v6: demand_buf is reconstructed from member time timestamps.

Benchmark (OTP 27.3, vs master baseline)

LIFO path overhead is within noise (0–5%) for all existing benchmarks; the cull-heavy
benchmark improves by ~8%. FIFO adds ~10–15% vs LIFO for single take/return
(queue vs list-head), negligible for drain-all and concurrent patterns.

Introduces two related changes:

1. `member_order => fifo | lifo` pool config option (default `lifo`,
   preserving existing stack behaviour). `fifo` uses an OTP `queue()`
   as the free-member structure, producing round-robin dispatch where
   the longest-idle worker is always handed out next.  The free-member
   abstraction (`free_push/pop/delete/fold/take_n/convert`) hides the
   list-vs-queue difference from all call sites; `free_count` is now
   exclusively managed by these helpers.

2. Pool-level demand-history cull replaces the per-worker `max_age`
   idle-time check.  A fixed-capacity OTP-queue ring buffer (head =
   current bucket, capacity = ceil(max_age/bucket_size)) tracks peak
   concurrent in-use count; at each cull tick the pool sizes to that
   peak (floored at `init_count`).  This gives LIFO and FIFO pools
   identical shrinkage behaviour for the same config, and correctly
   handles FIFO rotation where workers never accumulate idle time.

   `max_age` now means "demand memory window"; `cull_interval` default
   changed from {1,min} to {15,sec} so defaults satisfy CI <= MA.
   Hot-upgrade (vsn 5->6) reconstructs the demand buffer from member
   `time` timestamps.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/pooler.erl
update_demand_buf(NewInUse, Pool) ->
Now = erlang:monotonic_time(millisecond),
Q = Pool#pool.demand_buf,
{value, {CurMax, CurStart}} = queue:peek(Q),

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be rewritten to always use queue:out and in the true branch of the case below to use the original Q, but it won't give performance benefit because we keep current bucket on the queue head with in_r. The only time we may get an O(n) operation is when we call out_r, but it happens after we rotate len(Q) number of buckets, so amortized O(1) with no edge cases.

@seriyps seriyps merged commit bb2638c into epgsql:master Jun 2, 2026
14 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant