Skip to content

bench(transfer): par_bridge vs into_par_iter on 100K work items (#1403)#4200

Merged
oferchen merged 1 commit into
masterfrom
bench/par-bridge-vs-deque-1403
May 17, 2026
Merged

bench(transfer): par_bridge vs into_par_iter on 100K work items (#1403)#4200
oferchen merged 1 commit into
masterfrom
bench/par-bridge-vs-deque-1403

Conversation

@oferchen
Copy link
Copy Markdown
Owner

Summary

  • Adds crates/transfer/benches/par_bridge_vs_deque.rs, a Criterion bench that compares three rayon dispatch shapes on a synthetic 100K small-file workload: into_par_iter over an owned Vec, par_bridge over a pre-allocated Vec iterator, and par_bridge over a pure generator iterator.
  • Sweeps {1, 4, 8, 16} workers via a private rayon pool per row so the global pool size cannot skew results; throughput is reported in items/sec via Throughput::Elements(100_000).
  • Top-of-file doc spells out the hypothesis (the par_bridge channel becomes a bottleneck at 100K-file scales) and the action favourable or unfavourable numbers would inform for tasks Add coverage for burst-only bandwidth overrides #1284 (io_uring + rayon composition), Add brand parsing helpers for environment overrides #1370 (per-thread fold), and Refactor daemon CLI into dedicated module #1681 (gated dispatch refactor).

Notes

  • Bench only - no production code is touched.
  • Work items are pre-allocated outside the timed section.
  • Cross-platform: no platform-gated APIs, uses only std, criterion, and rayon.
  • Per-item closure is a deterministic FNV-1a fold sized to roughly mimic a small file fingerprint, kept short so the dispatch cost dominates and the shape difference shows up cleanly.

Test plan

Adds a Criterion bench in crates/transfer/benches that compares three
rayon dispatch shapes - `into_par_iter` over an owned Vec, `par_bridge`
over a Vec iterator, and `par_bridge` over a pure generator - across
{1, 4, 8, 16} workers on a fixed 100K small-file workload. Throughput
is reported in items/sec via `Throughput::Elements`. The numbers feed
the dispatch-shape decisions tracked in #1284, #1370, and #1681.
@oferchen oferchen merged commit 2e3889a into master May 17, 2026
40 checks passed
@oferchen oferchen deleted the bench/par-bridge-vs-deque-1403 branch May 17, 2026 06:53
oferchen added a commit that referenced this pull request May 18, 2026
Adds a Criterion bench in crates/transfer/benches that compares three
rayon dispatch shapes - `into_par_iter` over an owned Vec, `par_bridge`
over a Vec iterator, and `par_bridge` over a pure generator - across
{1, 4, 8, 16} workers on a fixed 100K small-file workload. Throughput
is reported in items/sec via `Throughput::Elements`. The numbers feed
the dispatch-shape decisions tracked in #1284, #1370, and #1681.
oferchen added a commit that referenced this pull request May 18, 2026
Adds a Criterion bench in crates/transfer/benches that compares three
rayon dispatch shapes - `into_par_iter` over an owned Vec, `par_bridge`
over a Vec iterator, and `par_bridge` over a pure generator - across
{1, 4, 8, 16} workers on a fixed 100K small-file workload. Throughput
is reported in items/sec via `Throughput::Elements`. The numbers feed
the dispatch-shape decisions tracked in #1284, #1370, and #1681.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant