Skip to content

feat(engine): BufferPool byte-budget memory cap (#2245)#4160

Merged
oferchen merged 1 commit into
masterfrom
feat/bufferpool-byte-budget-2245
May 16, 2026
Merged

feat(engine): BufferPool byte-budget memory cap (#2245)#4160
oferchen merged 1 commit into
masterfrom
feat/bufferpool-byte-budget-2245

Conversation

@oferchen
Copy link
Copy Markdown
Owner

Summary

  • BufferPool's count-only cap is insufficient when individual buffers vary widely in size: a handful of adaptive large-file buffers (e.g. 1 MiB each at ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count.
  • Adds an optional soft byte budget on pool retention, layered on top of the existing count cap.
  • Wires --max-alloc=N to the new byte budget (rather than the hard backpressure MemoryCap) so the flag bounds pool retention without blocking transfers.
  • Introduces a new pool-overflow counter exposed via BufferPool::total_byte_overflows() and on BufferPoolStats.

Cap strategy

Hybrid min(count_cap, byte_cap): a buffer is admitted to the central pool only when both the count slot and the byte budget have room. Either limit rejects independently. Justification:

  • The count cap continues to bound the number of ArrayQueue slots, which is correct at default buffer sizes (128 KiB) and well-tested.
  • The byte cap targets the failure mode the count cap cannot express: small numbers of large buffers blowing past the budget.
  • The two checks compose orthogonally with negligible runtime cost (one extra atomic CAS on return when a budget is set).
  • Backwards compatible: callers without a byte budget see identical behaviour.

Overflow counter

ByteBudget increments an atomic overflows counter every time try_reserve rejects an admission. Exposed via:

  • BufferPool::total_byte_overflows() -> u64
  • BufferPoolStats::total_byte_overflows
  • The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now includes byte_overflows=.

The byte-budget rejection path also deallocates rather than blocks, so the transfer pipeline never stalls when the budget is exhausted - subsequent acquires allocate fresh outside the pool. The existing hard MemoryCap (with condvar backpressure) remains available for callers that explicitly opt in via with_memory_cap.

CLI wiring

--max-alloc=N previously fed the hard MemoryCap. It now feeds the soft byte budget instead:

  • crates/core/src/client/run/mod.rs::apply_max_alloc populates GlobalBufferPoolConfig::byte_budget.
  • crates/cli/src/frontend/server/run.rs (server mode equivalent) does the same.

Matches user intent: bound the memory the buffer pool retains across the run rather than block the transfer when the cap is hit.

Test plan

  • cargo fmt --all -- --check
  • cargo clippy -p engine -p cli -p core --all-targets --all-features --no-deps -- -D warnings
  • New unit tests in byte_budget.rs (8 tests) cover try_reserve, release, overflow counter, saturating add, zero-limit panic.
  • New BufferPool unit tests in tests.rs cover: default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires.
  • Existing BufferPoolStats literal tests updated for new field.
  • CI nextest (Linux/macOS/Windows + musl).

The BufferPool count cap is insufficient when individual buffers vary
widely in size: a handful of adaptive large-file buffers (1 MiB each at
ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a
modest slot count. This change adds an optional soft byte budget on pool
retention, layered on top of the existing count cap.

Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the
central pool only when both the count slot and the byte budget have
room; either limit rejects independently. The count cap continues to
bound queue slots; the byte cap targets the failure mode the count cap
cannot express. The two checks compose orthogonally with negligible
runtime cost (one extra atomic CAS on return when a budget is set), and
callers without a byte budget see identical behaviour.

Overflow handling: when admission is rejected by the byte budget, the
buffer is deallocated and an atomic overflow counter increments. The
counter is exposed via BufferPool::total_byte_overflows() and on
BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now
includes byte_overflows=. Acquire never blocks on the byte budget -
on pool miss it allocates fresh.

CLI wiring: --max-alloc=N now feeds the soft byte budget rather than
the hard MemoryCap. Bounds pool retention without blocking transfers
when the cap is hit, matching user intent. The existing MemoryCap
(condvar backpressure) remains available for callers that opt in via
with_memory_cap.

Tests: unit tests in byte_budget.rs cover try_reserve, release,
overflow counter, saturating add, zero-limit panic. New BufferPool
tests in tests.rs cover default unset, builder sets, allows below cap,
falls through to direct alloc at cap, counter accumulates, capacity
recycles after acquire, min-of-both with count cap, stats field
exposed, panic on zero, does not block acquires. Existing
BufferPoolStats literal tests updated for the new field.
@github-actions github-actions Bot added the enhancement New feature or request label May 16, 2026
@oferchen oferchen merged commit 6a615aa into master May 16, 2026
45 checks passed
@oferchen oferchen deleted the feat/bufferpool-byte-budget-2245 branch May 16, 2026 16:53
oferchen added a commit that referenced this pull request May 18, 2026
The BufferPool count cap is insufficient when individual buffers vary
widely in size: a handful of adaptive large-file buffers (1 MiB each at
ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a
modest slot count. This change adds an optional soft byte budget on pool
retention, layered on top of the existing count cap.

Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the
central pool only when both the count slot and the byte budget have
room; either limit rejects independently. The count cap continues to
bound queue slots; the byte cap targets the failure mode the count cap
cannot express. The two checks compose orthogonally with negligible
runtime cost (one extra atomic CAS on return when a budget is set), and
callers without a byte budget see identical behaviour.

Overflow handling: when admission is rejected by the byte budget, the
buffer is deallocated and an atomic overflow counter increments. The
counter is exposed via BufferPool::total_byte_overflows() and on
BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now
includes byte_overflows=. Acquire never blocks on the byte budget -
on pool miss it allocates fresh.

CLI wiring: --max-alloc=N now feeds the soft byte budget rather than
the hard MemoryCap. Bounds pool retention without blocking transfers
when the cap is hit, matching user intent. The existing MemoryCap
(condvar backpressure) remains available for callers that opt in via
with_memory_cap.

Tests: unit tests in byte_budget.rs cover try_reserve, release,
overflow counter, saturating add, zero-limit panic. New BufferPool
tests in tests.rs cover default unset, builder sets, allows below cap,
falls through to direct alloc at cap, counter accumulates, capacity
recycles after acquire, min-of-both with count cap, stats field
exposed, panic on zero, does not block acquires. Existing
BufferPoolStats literal tests updated for the new field.
oferchen added a commit that referenced this pull request May 18, 2026
The BufferPool count cap is insufficient when individual buffers vary
widely in size: a handful of adaptive large-file buffers (1 MiB each at
ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a
modest slot count. This change adds an optional soft byte budget on pool
retention, layered on top of the existing count cap.

Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the
central pool only when both the count slot and the byte budget have
room; either limit rejects independently. The count cap continues to
bound queue slots; the byte cap targets the failure mode the count cap
cannot express. The two checks compose orthogonally with negligible
runtime cost (one extra atomic CAS on return when a budget is set), and
callers without a byte budget see identical behaviour.

Overflow handling: when admission is rejected by the byte budget, the
buffer is deallocated and an atomic overflow counter increments. The
counter is exposed via BufferPool::total_byte_overflows() and on
BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now
includes byte_overflows=. Acquire never blocks on the byte budget -
on pool miss it allocates fresh.

CLI wiring: --max-alloc=N now feeds the soft byte budget rather than
the hard MemoryCap. Bounds pool retention without blocking transfers
when the cap is hit, matching user intent. The existing MemoryCap
(condvar backpressure) remains available for callers that opt in via
with_memory_cap.

Tests: unit tests in byte_budget.rs cover try_reserve, release,
overflow counter, saturating add, zero-limit panic. New BufferPool
tests in tests.rs cover default unset, builder sets, allows below cap,
falls through to direct alloc at cap, counter accumulates, capacity
recycles after acquire, min-of-both with count cap, stats field
exposed, panic on zero, does not block acquires. Existing
BufferPoolStats literal tests updated for the new field.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant