Skip to content

perf(engine): BufferPool byte-budget cap to bound memory under large buffers#4102

Closed
oferchen wants to merge 4 commits into
masterfrom
perf/buffer-pool-byte-budget-cap-2245
Closed

perf(engine): BufferPool byte-budget cap to bound memory under large buffers#4102
oferchen wants to merge 4 commits into
masterfrom
perf/buffer-pool-byte-budget-cap-2245

Conversation

@oferchen
Copy link
Copy Markdown
Owner

Summary

  • Add optional pooled_bytes_budget field to BufferPool that caps the total capacity() of buffers retained in the central queue.
  • Hybrid with existing count-based soft cap: whichever fires first prevents admission. Defends against very-large adaptive buffers (e.g. 1 MB ADAPTIVE_BUFFER_HUGE) blowing through the count-only ceiling.
  • Threaded through GlobalBufferPoolConfig::pooled_bytes_budget; default None preserves the historical count-only behavior so existing callers see no change.
  • Atomic pooled_bytes counter updated on every admit and pop; relaxed ordering keeps the overhead at one extra add/sub per central-queue operation.

Refs #2245.

Test plan

  • pooled_bytes_budget_drops_oversized_buffers: 1 MB buffers + 16 KB budget; central queue stays empty and pooled bytes stay within budget.
  • pooled_bytes_budget_high_retains_normally: generous budget; count cap fires first and pool fills its slots.
  • count_cap_still_enforced_without_byte_budget: regression test for existing count-only path.
  • pooled_bytes_tracks_admission_and_pop: pooled-byte counter increments on admit and decrements on pop.
  • pooled_bytes_budget_rejects_zero: with_pooled_bytes_budget(0) panics.
  • pooled_bytes_budget_zero_is_treated_as_unbounded in global.rs: filter logic mirrors memory_cap=Some(0).

…buffers

Adds an optional `pooled_bytes_budget` field to `BufferPool` that caps the
total `capacity()` of buffers retained in the central queue. The hybrid
cap with the existing count-based soft capacity defends against a handful
of very-large adaptive buffers blowing through the memory ceiling.

Threaded through `GlobalBufferPoolConfig::pooled_bytes_budget` so callers
opt in via the existing builder; default `None` preserves count-only
behavior with no regression for existing callers.

Refs #2245
@oferchen
Copy link
Copy Markdown
Owner Author

Tests reveal an accounting bug in the agent's pooled_bytes tracking (test pooled_bytes_tracks_admission_and_pop expects 0 but gets 1). Multiple test failures across platforms after the first fmt round. Closing for rework; task #2245 reopened.

@oferchen oferchen closed this May 15, 2026
@oferchen oferchen deleted the perf/buffer-pool-byte-budget-cap-2245 branch May 16, 2026 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant