feat(transfer): instrument prepare_pending_acl cost (INC_RECURSE I5)#4142
Merged
Conversation
Adds INC_RECURSE diagnostic counter I5 (#2200): two process-global AtomicU64 counters that track every prepare_pending_acl invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_prepare_acl helper bumped from prepare_pending_acl itself, which wraps the existing body in an Instant::now() pair. The early-return path (entry.is_symlink() or !flags.acls) is still counted so per-segment call rate stays comparable to entry count, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via prepare_acl_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point) and PR #4121 (I3 flush rate), and emitted via: - debug_log!(Genr, 1, "generator prepare_pending_acl totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::prepare_acl when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - prepare_acl_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_prepare_acl invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - prepare_acl_totals_observable_without_prep asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening prep call must return identical values. No new CLI flag, no wire-format change, no change to the public prepare_pending_acl signature. Closes #2200
4 tasks
oferchen
added a commit
that referenced
this pull request
May 16, 2026
…RSE I2) (#4143) Adds INC_RECURSE diagnostic counter I2 (#2197): two process-global AtomicU64 counters that track every encode_and_send_segment invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_segment_dispatch helper bumped from encode_and_send_segment itself, which wraps the existing body in an Instant::now() pair. The zero-entry early-return path is still counted so per-transfer dispatch rate stays comparable to segment scheduler activity, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via segment_dispatch_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point), PR #4121 (I3 flush rate) and PR #4142 (I5 prepare_pending_acl), and emitted via: - debug_log!(Genr, 1, "generator encode_and_send_segment totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::segment_dispatch when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - segment_dispatch_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_segment_dispatch invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - segment_dispatch_totals_observable_without_dispatch asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening dispatch call must return identical values. No new CLI flag, no wire-format change, no change to the public encode_and_send_segment signature. Closes #2197
Merged
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…4142) * feat(transfer): instrument prepare_pending_acl cost (INC_RECURSE I5) Adds INC_RECURSE diagnostic counter I5 (#2200): two process-global AtomicU64 counters that track every prepare_pending_acl invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_prepare_acl helper bumped from prepare_pending_acl itself, which wraps the existing body in an Instant::now() pair. The early-return path (entry.is_symlink() or !flags.acls) is still counted so per-segment call rate stays comparable to entry count, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via prepare_acl_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point) and PR #4121 (I3 flush rate), and emitted via: - debug_log!(Genr, 1, "generator prepare_pending_acl totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::prepare_acl when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - prepare_acl_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_prepare_acl invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - prepare_acl_totals_observable_without_prep asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening prep call must return identical values. No new CLI flag, no wire-format change, no change to the public prepare_pending_acl signature. Closes #2200 * fix(transfer): rewrap rustdoc list items for prepare_pending_acl counters
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…RSE I2) (#4143) Adds INC_RECURSE diagnostic counter I2 (#2197): two process-global AtomicU64 counters that track every encode_and_send_segment invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_segment_dispatch helper bumped from encode_and_send_segment itself, which wraps the existing body in an Instant::now() pair. The zero-entry early-return path is still counted so per-transfer dispatch rate stays comparable to segment scheduler activity, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via segment_dispatch_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point), PR #4121 (I3 flush rate) and PR #4142 (I5 prepare_pending_acl), and emitted via: - debug_log!(Genr, 1, "generator encode_and_send_segment totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::segment_dispatch when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - segment_dispatch_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_segment_dispatch invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - segment_dispatch_totals_observable_without_dispatch asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening dispatch call must return identical values. No new CLI flag, no wire-format change, no change to the public encode_and_send_segment signature. Closes #2197
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…4142) * feat(transfer): instrument prepare_pending_acl cost (INC_RECURSE I5) Adds INC_RECURSE diagnostic counter I5 (#2200): two process-global AtomicU64 counters that track every prepare_pending_acl invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_prepare_acl helper bumped from prepare_pending_acl itself, which wraps the existing body in an Instant::now() pair. The early-return path (entry.is_symlink() or !flags.acls) is still counted so per-segment call rate stays comparable to entry count, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via prepare_acl_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point) and PR #4121 (I3 flush rate), and emitted via: - debug_log!(Genr, 1, "generator prepare_pending_acl totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::prepare_acl when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - prepare_acl_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_prepare_acl invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - prepare_acl_totals_observable_without_prep asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening prep call must return identical values. No new CLI flag, no wire-format change, no change to the public prepare_pending_acl signature. Closes #2200 * fix(transfer): rewrap rustdoc list items for prepare_pending_acl counters
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…RSE I2) (#4143) Adds INC_RECURSE diagnostic counter I2 (#2197): two process-global AtomicU64 counters that track every encode_and_send_segment invocation on the generator file-list path and the cumulative wall time spent inside the helper across all generator transfers. The counters fire from a single record_segment_dispatch helper bumped from encode_and_send_segment itself, which wraps the existing body in an Instant::now() pair. The zero-entry early-return path is still counted so per-transfer dispatch rate stays comparable to segment scheduler activity, but contributes a near-zero ns delta. Totals are sampled in GeneratorContext::run via segment_dispatch_totals() in the same finalize block used by PR #4103 (I1 first-byte latency), PR #4120 (I4 NDX partition_point), PR #4121 (I3 flush rate) and PR #4142 (I5 prepare_pending_acl), and emitted via: - debug_log!(Genr, 1, "generator encode_and_send_segment totals: \ calls=N elapsed_ns=M") - tracing::debug! event on rsync::generator::segment_dispatch when the optional tracing feature is enabled Two unit tests in crates/transfer/src/generator/tests.rs cover the new counters: - segment_dispatch_call_counter_increments asserts the call counter and the cumulative elapsed_ns counter grow by at least N and sum-of-durations after N synthetic record_segment_dispatch invocations. The assertion uses >= because the counter is shared across the process and other tests may run concurrently. - segment_dispatch_totals_observable_without_dispatch asserts the totals snapshot is a pure read - constructing a generator does not bump the counter on its own, so two adjacent reads with no intervening dispatch call must return identical values. No new CLI flag, no wire-format change, no change to the public encode_and_send_segment signature. Closes #2197
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AtomicU64counters that track everyprepare_pending_aclinvocation on the generator file-list path and the cumulative wall time spent inside the helper.record_prepare_aclhelper bumped fromprepare_pending_aclitself, which wraps the existing body in anInstant::now()pair. The early-return path (entry.is_symlink()or!flags.acls) is still counted so per-segment call rate stays comparable to entry count, but contributes a near-zero ns delta.GeneratorContext::runviaprepare_acl_totals()in the same finalize block used by PR feat(generator): instrument first-byte latency in send_file_list (INC_RECURSE I1) #4103 (I1 first-byte latency), PR feat(transfer): add wire_to_flat_ndx partition_point counters (INC_RECURSE I4) #4120 (I4 NDX partition_point) and PR feat(transfer): instrument writer.flush rate (INC_RECURSE I3) #4121 (I3 flush rate), and emitted viadebug_log!(Genr, 1, ...)plus atracing::debug!event onrsync::generator::prepare_aclwhen the optionaltracingfeature is enabled.prepare_pending_aclsignature.Test plan
cargo nextest run -p transfer --all-features -E 'test(prepare_acl)'covers both new unit tests.--acls -vv --debug=GENRand confirm the newgenerator prepare_pending_acl totals: calls=N elapsed_ns=Mline is emitted at end-of-transfer.Closes #2200