Skip to content

feat(transfer): add ReorderBuffer stall-duration and queue-depth metrics#3984

Merged
oferchen merged 1 commit into
masterfrom
feat/reorder-buffer-metrics-1885
May 13, 2026
Merged

feat(transfer): add ReorderBuffer stall-duration and queue-depth metrics#3984
oferchen merged 1 commit into
masterfrom
feat/reorder-buffer-metrics-1885

Conversation

@oferchen
Copy link
Copy Markdown
Owner

Summary

  • Add ReorderBufferStats struct with five counters: current_depth, peak_depth, stall_count, total_stall_nanos, and items_delivered to the BoundedReorderBuffer in the transfer crate
  • Instrument insert() to track stall episodes (head-of-line blocking when a non-head item arrives into an empty buffer) and queue depth high-water marks
  • Add metrics() accessor returning a snapshot of all counters, plus mean_stall_nanos() derived accessor
  • Support deterministic clock injection via with_clock() constructor for reproducible test assertions on stall timing

Implements the observability layer from the design note in docs/design/reorderbuffer-metrics-and-bypass.md (Part A). This is a prerequisite for the bypass optimization (#1886) - the metrics confirm whether head-of-line stalls are significant enough to justify bypassing the reorder buffer when --delay-updates is off.

Test plan

  • in_order_delivery_no_stall - verifies zero stalls and zero stall time for sequential insertion
  • out_of_order_produces_stall - verifies stall count increments and stall duration accumulates when head is missing
  • multiple_stall_episodes - verifies distinct stall episodes are counted independently
  • peak_depth_monotonically_nondecreasing - verifies peak depth never decreases across insertions and drains
  • queue_depth_tracks_buffered_count - verifies current_depth matches buffered_count()
  • items_delivered_counts_all_drained - verifies delivery counter accuracy
  • mean_stall_nanos_derived_correctly - verifies arithmetic on the derived accessor
  • stale_insert_does_not_affect_metrics - verifies already-delivered sequence numbers do not perturb counters
  • Property test: counters_monotonic_across_permutation - all counters are monotonically non-decreasing across reversed permutations
  • Property test: identity_permutation_zero_stalls - identity permutation always produces zero stalls
  • All 25 existing reorder buffer tests continue to pass

@github-actions github-actions Bot added the enhancement New feature or request label May 13, 2026
@oferchen oferchen merged commit 68da13d into master May 13, 2026
39 checks passed
@oferchen oferchen deleted the feat/reorder-buffer-metrics-1885 branch May 14, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant