Prune stale fallback cache entries#2245
Merged
Merged
Conversation
6 tasks
oferchen
added a commit
that referenced
this pull request
May 16, 2026
The BufferPool count cap is insufficient when individual buffers vary widely in size: a handful of adaptive large-file buffers (1 MiB each at ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count. This change adds an optional soft byte budget on pool retention, layered on top of the existing count cap. Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the central pool only when both the count slot and the byte budget have room; either limit rejects independently. The count cap continues to bound queue slots; the byte cap targets the failure mode the count cap cannot express. The two checks compose orthogonally with negligible runtime cost (one extra atomic CAS on return when a budget is set), and callers without a byte budget see identical behaviour. Overflow handling: when admission is rejected by the byte budget, the buffer is deallocated and an atomic overflow counter increments. The counter is exposed via BufferPool::total_byte_overflows() and on BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now includes byte_overflows=. Acquire never blocks on the byte budget - on pool miss it allocates fresh. CLI wiring: --max-alloc=N now feeds the soft byte budget rather than the hard MemoryCap. Bounds pool retention without blocking transfers when the cap is hit, matching user intent. The existing MemoryCap (condvar backpressure) remains available for callers that opt in via with_memory_cap. Tests: unit tests in byte_budget.rs cover try_reserve, release, overflow counter, saturating add, zero-limit panic. New BufferPool tests in tests.rs cover default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires. Existing BufferPoolStats literal tests updated for the new field.
3 tasks
oferchen
added a commit
that referenced
this pull request
May 17, 2026
Consolidates issues #1271 and #1370 into a single design proposal for promoting the engine BufferPool's thread-local cache from a single slot in front of a shared ArrayQueue to a per-thread slab as the primary storage, with the existing ArrayQueue demoted to a bounded global overflow / balancer. Covers: contrast with sharded-mutex (#1295) and per-thread-cache (#1370) alternatives, LIFO slab structure with cross-thread return handling, bounded memory accounting layered on top of #2245's byte budget, preserved PooledBuffer Drop and BufferAllocator surface, failure modes (thread teardown, panicking thread, long-lived buffers), comparison table, trigger conditions for adoption, and a five-step implementation sequence. Recommendation is to defer implementation until profiling at 32+ sustained threads proves the existing two-level layout actually contends.
4 tasks
oferchen
added a commit
that referenced
this pull request
May 18, 2026
The BufferPool count cap is insufficient when individual buffers vary widely in size: a handful of adaptive large-file buffers (1 MiB each at ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count. This change adds an optional soft byte budget on pool retention, layered on top of the existing count cap. Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the central pool only when both the count slot and the byte budget have room; either limit rejects independently. The count cap continues to bound queue slots; the byte cap targets the failure mode the count cap cannot express. The two checks compose orthogonally with negligible runtime cost (one extra atomic CAS on return when a budget is set), and callers without a byte budget see identical behaviour. Overflow handling: when admission is rejected by the byte budget, the buffer is deallocated and an atomic overflow counter increments. The counter is exposed via BufferPool::total_byte_overflows() and on BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now includes byte_overflows=. Acquire never blocks on the byte budget - on pool miss it allocates fresh. CLI wiring: --max-alloc=N now feeds the soft byte budget rather than the hard MemoryCap. Bounds pool retention without blocking transfers when the cap is hit, matching user intent. The existing MemoryCap (condvar backpressure) remains available for callers that opt in via with_memory_cap. Tests: unit tests in byte_budget.rs cover try_reserve, release, overflow counter, saturating add, zero-limit panic. New BufferPool tests in tests.rs cover default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires. Existing BufferPoolStats literal tests updated for the new field.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
Consolidates issues #1271 and #1370 into a single design proposal for promoting the engine BufferPool's thread-local cache from a single slot in front of a shared ArrayQueue to a per-thread slab as the primary storage, with the existing ArrayQueue demoted to a bounded global overflow / balancer. Covers: contrast with sharded-mutex (#1295) and per-thread-cache (#1370) alternatives, LIFO slab structure with cross-thread return handling, bounded memory accounting layered on top of #2245's byte budget, preserved PooledBuffer Drop and BufferAllocator surface, failure modes (thread teardown, panicking thread, long-lived buffers), comparison table, trigger conditions for adoption, and a five-step implementation sequence. Recommendation is to defer implementation until profiling at 32+ sustained threads proves the existing two-level layout actually contends.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
The BufferPool count cap is insufficient when individual buffers vary widely in size: a handful of adaptive large-file buffers (1 MiB each at ADAPTIVE_BUFFER_HUGE) blow past any reasonable memory budget even with a modest slot count. This change adds an optional soft byte budget on pool retention, layered on top of the existing count cap. Strategy: hybrid min(count_cap, byte_cap). A buffer is admitted to the central pool only when both the count slot and the byte budget have room; either limit rejects independently. The count cap continues to bound queue slots; the byte cap targets the failure mode the count cap cannot express. The two checks compose orthogonally with negligible runtime cost (one extra atomic CAS on return when a budget is set), and callers without a byte budget see identical behaviour. Overflow handling: when admission is rejected by the byte budget, the buffer is deallocated and an atomic overflow counter increments. The counter is exposed via BufferPool::total_byte_overflows() and on BufferPoolStats. The OC_RSYNC_BUFFER_POOL_STATS=1 Drop print now includes byte_overflows=. Acquire never blocks on the byte budget - on pool miss it allocates fresh. CLI wiring: --max-alloc=N now feeds the soft byte budget rather than the hard MemoryCap. Bounds pool retention without blocking transfers when the cap is hit, matching user intent. The existing MemoryCap (condvar backpressure) remains available for callers that opt in via with_memory_cap. Tests: unit tests in byte_budget.rs cover try_reserve, release, overflow counter, saturating add, zero-limit panic. New BufferPool tests in tests.rs cover default unset, builder sets, allows below cap, falls through to direct alloc at cap, counter accumulates, capacity recycles after acquire, min-of-both with count cap, stats field exposed, panic on zero, does not block acquires. Existing BufferPoolStats literal tests updated for the new field.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
Consolidates issues #1271 and #1370 into a single design proposal for promoting the engine BufferPool's thread-local cache from a single slot in front of a shared ArrayQueue to a per-thread slab as the primary storage, with the existing ArrayQueue demoted to a bounded global overflow / balancer. Covers: contrast with sharded-mutex (#1295) and per-thread-cache (#1370) alternatives, LIFO slab structure with cross-thread return handling, bounded memory accounting layered on top of #2245's byte budget, preserved PooledBuffer Drop and BufferAllocator surface, failure modes (thread teardown, panicking thread, long-lived buffers), comparison table, trigger conditions for adoption, and a five-step implementation sequence. Recommendation is to defer implementation until profiling at 32+ sustained threads proves the existing two-level layout actually contends.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Testing
Codex Task