feat(engine): BufferPool byte-budget memory cap (#2245)#4160
Merged
Conversation
The BufferPool count cap is insufficient when individual buffers vary widely in size: a handful of adaptive large-file buffers (1 MiB each at ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count. This change adds an optional soft byte budget on pool retention, layered on top of the existing count cap. Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the central pool only when both the count slot and the byte budget have room; either limit rejects independently. The count cap continues to bound queue slots; the byte cap targets the failure mode the count cap cannot express. The two checks compose orthogonally with negligible runtime cost (one extra atomic CAS on return when a budget is set), and callers without a byte budget see identical behaviour. Overflow handling: when admission is rejected by the byte budget, the buffer is deallocated and an atomic overflow counter increments. The counter is exposed via BufferPool::total_byte_overflows() and on BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now includes byte_overflows=. Acquire never blocks on the byte budget - on pool miss it allocates fresh. CLI wiring: --max-alloc=N now feeds the soft byte budget rather than the hard MemoryCap. Bounds pool retention without blocking transfers when the cap is hit, matching user intent. The existing MemoryCap (condvar backpressure) remains available for callers that opt in via with_memory_cap. Tests: unit tests in byte_budget.rs cover try_reserve, release, overflow counter, saturating add, zero-limit panic. New BufferPool tests in tests.rs cover default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires. Existing BufferPoolStats literal tests updated for the new field.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
The BufferPool count cap is insufficient when individual buffers vary widely in size: a handful of adaptive large-file buffers (1 MiB each at ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count. This change adds an optional soft byte budget on pool retention, layered on top of the existing count cap. Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the central pool only when both the count slot and the byte budget have room; either limit rejects independently. The count cap continues to bound queue slots; the byte cap targets the failure mode the count cap cannot express. The two checks compose orthogonally with negligible runtime cost (one extra atomic CAS on return when a budget is set), and callers without a byte budget see identical behaviour. Overflow handling: when admission is rejected by the byte budget, the buffer is deallocated and an atomic overflow counter increments. The counter is exposed via BufferPool::total_byte_overflows() and on BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now includes byte_overflows=. Acquire never blocks on the byte budget - on pool miss it allocates fresh. CLI wiring: --max-alloc=N now feeds the soft byte budget rather than the hard MemoryCap. Bounds pool retention without blocking transfers when the cap is hit, matching user intent. The existing MemoryCap (condvar backpressure) remains available for callers that opt in via with_memory_cap. Tests: unit tests in byte_budget.rs cover try_reserve, release, overflow counter, saturating add, zero-limit panic. New BufferPool tests in tests.rs cover default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires. Existing BufferPoolStats literal tests updated for the new field.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
The BufferPool count cap is insufficient when individual buffers vary widely in size: a handful of adaptive large-file buffers (1 MiB each at ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count. This change adds an optional soft byte budget on pool retention, layered on top of the existing count cap. Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the central pool only when both the count slot and the byte budget have room; either limit rejects independently. The count cap continues to bound queue slots; the byte cap targets the failure mode the count cap cannot express. The two checks compose orthogonally with negligible runtime cost (one extra atomic CAS on return when a budget is set), and callers without a byte budget see identical behaviour. Overflow handling: when admission is rejected by the byte budget, the buffer is deallocated and an atomic overflow counter increments. The counter is exposed via BufferPool::total_byte_overflows() and on BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now includes byte_overflows=. Acquire never blocks on the byte budget - on pool miss it allocates fresh. CLI wiring: --max-alloc=N now feeds the soft byte budget rather than the hard MemoryCap. Bounds pool retention without blocking transfers when the cap is hit, matching user intent. The existing MemoryCap (condvar backpressure) remains available for callers that opt in via with_memory_cap. Tests: unit tests in byte_budget.rs cover try_reserve, release, overflow counter, saturating add, zero-limit panic. New BufferPool tests in tests.rs cover default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires. Existing BufferPoolStats literal tests updated for the new field.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count.--max-alloc=Nto the new byte budget (rather than the hard backpressureMemoryCap) so the flag bounds pool retention without blocking transfers.BufferPool::total_byte_overflows()and onBufferPoolStats.Cap strategy
Hybrid
min(count_cap, byte_cap): a buffer is admitted to the central pool only when both the count slot and the byte budget have room. Either limit rejects independently. Justification:ArrayQueueslots, which is correct at default buffer sizes (128 KiB) and well-tested.Overflow counter
ByteBudgetincrements an atomicoverflowscounter every timetry_reserverejects an admission. Exposed via:BufferPool::total_byte_overflows() -> u64BufferPoolStats::total_byte_overflowsOC_RSYNC_BUFFER_POOL_STATS=1Drop print now includesbyte_overflows=.The byte-budget rejection path also deallocates rather than blocks, so the transfer pipeline never stalls when the budget is exhausted - subsequent acquires allocate fresh outside the pool. The existing hard
MemoryCap(with condvar backpressure) remains available for callers that explicitly opt in viawith_memory_cap.CLI wiring
--max-alloc=Npreviously fed the hardMemoryCap. It now feeds the soft byte budget instead:crates/core/src/client/run/mod.rs::apply_max_allocpopulatesGlobalBufferPoolConfig::byte_budget.crates/cli/src/frontend/server/run.rs(server mode equivalent) does the same.Matches user intent: bound the memory the buffer pool retains across the run rather than block the transfer when the cap is hit.
Test plan
cargo fmt --all -- --checkcargo clippy -p engine -p cli -p core --all-targets --all-features --no-deps -- -D warningsbyte_budget.rs(8 tests) covertry_reserve,release, overflow counter, saturating add, zero-limit panic.BufferPoolunit tests intests.rscover: default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires.BufferPoolStatsliteral tests updated for new field.