feat(transfer): instrument encode_and_send_segment dispatch (INC_RECURSE I2)#4143
Merged
Merged
Conversation
…RSE I2) Adds INC_RECURSE diagnostic counter I2 (#2197): two process-global AtomicU64 counters that track every encode_and_send_segment invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_segment_dispatch helper bumped from encode_and_send_segment itself, which wraps the existing body in an Instant::now() pair. The zero-entry early-return path is still counted so per-transfer dispatch rate stays comparable to segment scheduler activity, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via segment_dispatch_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point), PR #4121 (I3 flush rate) and PR #4142 (I5 prepare_pending_acl), and emitted via: - debug_log!(Genr, 1, "generator encode_and_send_segment totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::segment_dispatch when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - segment_dispatch_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_segment_dispatch invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - segment_dispatch_totals_observable_without_dispatch asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening dispatch call must return identical values. No new CLI flag, no wire-format change, no change to the public encode_and_send_segment signature. Closes #2197
Merged
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…RSE I2) (#4143) Adds INC_RECURSE diagnostic counter I2 (#2197): two process-global AtomicU64 counters that track every encode_and_send_segment invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_segment_dispatch helper bumped from encode_and_send_segment itself, which wraps the existing body in an Instant::now() pair. The zero-entry early-return path is still counted so per-transfer dispatch rate stays comparable to segment scheduler activity, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via segment_dispatch_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point), PR #4121 (I3 flush rate) and PR #4142 (I5 prepare_pending_acl), and emitted via: - debug_log!(Genr, 1, "generator encode_and_send_segment totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::segment_dispatch when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - segment_dispatch_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_segment_dispatch invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - segment_dispatch_totals_observable_without_dispatch asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening dispatch call must return identical values. No new CLI flag, no wire-format change, no change to the public encode_and_send_segment signature. Closes #2197
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…RSE I2) (#4143) Adds INC_RECURSE diagnostic counter I2 (#2197): two process-global AtomicU64 counters that track every encode_and_send_segment invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_segment_dispatch helper bumped from encode_and_send_segment itself, which wraps the existing body in an Instant::now() pair. The zero-entry early-return path is still counted so per-transfer dispatch rate stays comparable to segment scheduler activity, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via segment_dispatch_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point), PR #4121 (I3 flush rate) and PR #4142 (I5 prepare_pending_acl), and emitted via: - debug_log!(Genr, 1, "generator encode_and_send_segment totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::segment_dispatch when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - segment_dispatch_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_segment_dispatch invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - segment_dispatch_totals_observable_without_dispatch asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening dispatch call must return identical values. No new CLI flag, no wire-format change, no change to the public encode_and_send_segment signature. Closes #2197
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AtomicU64counters that track everyencode_and_send_segmentinvocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers.record_segment_dispatchhelper bumped fromencode_and_send_segmentitself; the timer wraps the body viaInstant::now(), so the zero-entry early-return path is still counted while contributing a near-zero ns delta.GeneratorContext::runviasegment_dispatch_totals()in the same finalize block used by PR feat(generator): instrument first-byte latency in send_file_list (INC_RECURSE I1) #4103 (I1 first-byte latency), PR feat(transfer): add wire_to_flat_ndx partition_point counters (INC_RECURSE I4) #4120 (I4 NDXpartition_point), PR feat(transfer): instrument writer.flush rate (INC_RECURSE I3) #4121 (I3 flush rate) and PR feat(transfer): instrument prepare_pending_acl cost (INC_RECURSE I5) #4142 (I5prepare_pending_acl), and emitted viadebug_log!(Genr, 1, ...)plus an optionaltracing::debug!event onrsync::generator::segment_dispatch.No new CLI flag, no wire-format change, no change to the public
encode_and_send_segmentsignature.Test plan
cargo fmt --allsegment_dispatch_call_counter_incrementsandsegment_dispatch_totals_observable_without_dispatchincrates/transfer/src/generator/tests.rsCloses #2197