Skip to content

docs(transfer): document ReorderBuffer head-of-line blocking semantics (#1883)#3519

Merged
oferchen merged 2 commits into
masterfrom
docs/reorder-buffer-hol-1883
May 1, 2026
Merged

docs(transfer): document ReorderBuffer head-of-line blocking semantics (#1883)#3519
oferchen merged 2 commits into
masterfrom
docs/reorder-buffer-hol-1883

Conversation

@oferchen
Copy link
Copy Markdown
Owner

@oferchen oferchen commented May 1, 2026

Summary

Adds docs/architecture/reorder-buffer.md documenting the head-of-line
(HoL) blocking behaviour of the concurrent delta pipeline's reorder buffer
and contrasting it against upstream rsync 3.4.1's strictly-sequential
per-file pipeline.

Tracker: oc-rsync task #1883.

The doc covers:

  • Both reorder-buffer implementations and where they sit in the pipeline:
    the ring-buffer ReorderBuffer (crates/engine/src/concurrent_delta/reorder.rs:65)
    owned by the delta-reorder consumer thread
    (crates/engine/src/concurrent_delta/consumer.rs:147-188), and the
    alternative sliding-window BoundedReorderBuffer
    (crates/transfer/src/reorder_buffer.rs:57) with its explicit
    BackpressureError.
  • Current HoL semantics: a slow file at next_expected stalls every
    completed successor in the bounded window (default
    2 * rayon::current_num_threads() slots), and force_insert
    (reorder.rs:334) trades the memory bound for forward progress when
    the head slot is empty and the window fills.
  • Trade-offs vs upstream's recv_files loop
    (target/interop/upstream-src/rsync-3.4.1/receiver.c:522): higher
    throughput under uniform file sizes, but a "stop-then-burst" UX pattern
    on skewed distributions.
  • When HoL blocking is observable: mixed large-and-small workloads,
    --delay-updates runs (mirroring upstream's handle_delayed_updates at
    receiver.c:422), phase 2 redo, and the small-file regime where it is
    effectively invisible.
  • Pointers to related work-in-progress trackers: Ensure release workflow uploads architecture-specific assets #1884 (bounded-memory
    spill-to-tempfile for stalled successors), Ensure release task uploads cross-compiled binaries #1885 (stall-duration metrics),
    and Ensure release checksums are verifiable #1886 (bypass parallel pipeline when --delay-updates is off). All
    three are described as open design items, not merged code.

Documentation only - no code changes, no other docs cross-linked.

Test plan

  • git diff confirms only docs/architecture/reorder-buffer.md is added.
  • CI: fmt+clippy, nextest stable, Windows stable, macOS stable, Linux musl
    stable, interop. Docs-only PR; no code paths touched.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label May 1, 2026
@oferchen oferchen merged commit d28b4d9 into master May 1, 2026
11 checks passed
@oferchen oferchen deleted the docs/reorder-buffer-hol-1883 branch May 1, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant