Skip to content

feat: bounded-memory spill-to-tempfile for ReorderBuffer#3982

Merged
oferchen merged 1 commit into
masterfrom
feat/reorder-buffer-spill-1884
May 13, 2026
Merged

feat: bounded-memory spill-to-tempfile for ReorderBuffer#3982
oferchen merged 1 commit into
masterfrom
feat/reorder-buffer-spill-1884

Conversation

@oferchen
Copy link
Copy Markdown
Owner

Summary

  • Adds SpillableReorderBuffer<T> that wraps ReorderBuffer<T> with transparent disk-backed overflow when in-memory footprint exceeds a configurable threshold (default 64 MB)
  • Introduces SpillCodec trait for item serialization using a simple length-prefixed binary format, with implementation for DeltaResult
  • Adds ReorderBuffer::take() for non-advancing slot extraction, enabling the spill layer to extract items without disturbing delivery order

Design

When successor items accumulate in the reorder buffer due to head-of-line stalls (e.g., a slow delta computation blocks delivery of 100K+ subsequent items), memory grows unboundedly. This PR bounds memory by spilling the highest-sequence items (furthest from delivery) to a SpooledTempFile (in-memory up to 1 MB, then rolls to disk). A hot zone around next_expected is preserved in memory to avoid thrashing. Items are reloaded transparently when their sequence reaches the delivery cursor.

The spill format is a simple [u32 len][payload] record per item, indexed by BTreeMap<sequence, file_offset> for O(log S) reload. Temp files are cleaned up automatically via RAII (Drop).

Test plan

  • No spill when under threshold (zero serialization cost)
  • Spill triggers when threshold exceeded
  • Correct delivery order after spill and reload
  • Cleanup on drop (temp files removed)
  • Interleaved spill/deliver cycles
  • Exact threshold boundary (no spill at boundary, spill above)
  • Empty buffer operations
  • Force insert with spill
  • Spill stats tracking (spill/reload event counters)
  • Large-scale spill and drain (100 items)
  • DeltaResult SpillCodec roundtrip (Success, NeedsRedo, Failed)
  • SpillableReorderBuffer with DeltaResult items
  • ReorderBuffer::take() extracts without advancing cursor

Closes #1884

When the in-memory reorder buffer exceeds a configurable threshold
(default 64 MB), excess items are serialized to a temporary file and
reloaded transparently on delivery. This bounds memory for 100K+ file
transfers where head-of-line stalls cause successor accumulation.

Key components:
- SpillCodec trait for item serialization (length-prefixed binary)
- SpillableReorderBuffer<T> wrapper with same API as ReorderBuffer
- SpillCodec implementation for DeltaResult
- ReorderBuffer::take() for non-advancing slot extraction
- SpooledTempFile (in-memory up to 1 MB, then disk) via tempfile crate
- Hot-zone protection keeps items near next_expected in memory
- RAII cleanup: temp files removed automatically on drop
@github-actions github-actions Bot added the enhancement New feature or request label May 13, 2026
@oferchen oferchen merged commit c4f2c10 into master May 13, 2026
40 checks passed
@oferchen oferchen deleted the feat/reorder-buffer-spill-1884 branch May 13, 2026 11:06
oferchen added a commit that referenced this pull request May 18, 2026
…3982)

When the in-memory reorder buffer exceeds a configurable threshold
(default 64 MB), excess items are serialized to a temporary file and
reloaded transparently on delivery. This bounds memory for 100K+ file
transfers where head-of-line stalls cause successor accumulation.

Key components:
- SpillCodec trait for item serialization (length-prefixed binary)
- SpillableReorderBuffer<T> wrapper with same API as ReorderBuffer
- SpillCodec implementation for DeltaResult
- ReorderBuffer::take() for non-advancing slot extraction
- SpooledTempFile (in-memory up to 1 MB, then disk) via tempfile crate
- Hot-zone protection keeps items near next_expected in memory
- RAII cleanup: temp files removed automatically on drop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant