Skip to content

docs(audits): ReorderBuffer memory profile at 100K+ parallel results#3821

Merged
oferchen merged 1 commit into
masterfrom
docs/reorder-buffer-memory-1564
May 7, 2026
Merged

docs(audits): ReorderBuffer memory profile at 100K+ parallel results#3821
oferchen merged 1 commit into
masterfrom
docs/reorder-buffer-memory-1564

Conversation

@oferchen
Copy link
Copy Markdown
Owner

@oferchen oferchen commented May 7, 2026

Summary

  • Quantify per-entry memory cost (48-72 bytes inline; no Arc carry-over) for engine::concurrent_delta::ReorderBuffer and transfer::reorder_buffer::BoundedReorderBuffer.
  • Document worst-case memory at 100K / 1M / 10M stalled successors: 7.6 MiB / 80 MiB / 1+ GiB for the ring buffer, plus the BTreeMap variant comparison.
  • Identify peak-driver paths: bounded ring + force_insert escape hatch vs. BackpressureError window. Confirm spill-to-tempfile (Ensure release workflow uploads architecture-specific assets #1884) is not implemented today and the architectural doc still lists it as open.
  • Sketch a peak-RSS bench fixture extending test(transfer): ReorderBuffer benchmark at 10M+ file scale #3780's 10M scaling bench with allocator-level high-water sampling.

Test plan

  • Doc-only change. CI fmt + clippy + nextest pass.
  • Verify file paths and line numbers cited in the audit match crates/engine/src/concurrent_delta/{reorder.rs,types.rs,consumer.rs} and crates/transfer/src/reorder_buffer.rs.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label May 7, 2026
@oferchen oferchen merged commit 72c2cf8 into master May 7, 2026
8 checks passed
@oferchen oferchen deleted the docs/reorder-buffer-memory-1564 branch May 8, 2026 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant