Optimize rolling checksum bulk rolls#1933
Merged
Merged
Conversation
This was referenced May 1, 2026
oferchen
added a commit
that referenced
this pull request
May 5, 2026
Defines the empirical benchmark harness, workloads (100 / 1k / 10k concurrent clients), metrics, soft-limit triggers, comparison oracle against upstream rsync 3.4.1, and the decision criteria that gate the async listener migration tracked under #1935. Frames the active-counter fix from the parent audit (#1673, PR #3705) as a strict precondition. Tracking: oc-rsync task #1933.
oferchen
added a commit
that referenced
this pull request
May 5, 2026
) Defines the empirical benchmark harness, workloads (100 / 1k / 10k concurrent clients), metrics, soft-limit triggers, comparison oracle against upstream rsync 3.4.1, and the decision criteria that gate the async listener migration tracked under #1935. Frames the active-counter fix from the parent audit (#1673, PR #3705) as a strict precondition. Tracking: oc-rsync task #1933.
3 tasks
oferchen
added a commit
that referenced
this pull request
May 7, 2026
…#3891) Slim runnable plan complementing the broader benchmark plan. Specifies the minimum harness needed to land first measured numbers on Linux loopback at 100/1K/10K concurrent rsync:// pulls so #1934 RFC and #1935 async-listener work can compare sync vs async paths against quantified sync-baseline ttfb, completion p99, peak RSS, and thread count.
This was referenced May 14, 2026
oferchen
added a commit
that referenced
this pull request
May 16, 2026
#1933) (#4182) * chore(bench): stress harness for thread-per-connection scaling Adds an integration-test-shaped stress benchmark that drives 100, 1000, and 10000 concurrent TCP clients against the daemon listener, capturing wall time, ECONNREFUSED / EMFILE counts, and peak RSS via getrusage. All three scenarios are marked #[ignore]; the 10k case is unix-only and self-skips when RLIMIT_NOFILE cannot accommodate the request. The harness exists to provide evidence for whether an async listener (tracked separately) would be a meaningful change to the current std::thread::spawn-per-connection model. * style(daemon): apply rustfmt to connection_scaling_stress * fix(daemon): convert ru_maxrss via i64::from for cross-arch portability * fix(daemon): use ru_maxrss directly without conversion The conversion was redundant on all targets we build on: c_long is i64 on 64-bit Linux/macOS (no conversion needed) and i32 on 32-bit Linux (the subsequent 'as u64' sign-extends to i64 first, then saturating_mul operates on u64). Removing the conversion silences both clippy::unnecessary_cast and clippy::useless_conversion without losing overflow safety.
Merged
3 tasks
oferchen
added a commit
that referenced
this pull request
May 17, 2026
Add a focused evaluation of async runtime options for the daemon accept loop. Compares tokio against async-std and the existing thread-per-connection model, and records the decision to adopt tokio with the rt-multi-thread flavour under the existing async feature gate. The doc complements the implementation plan in #1935 and the benchmark plan in #1933 rather than restating either. It covers maintenance posture, feature parity, ecosystem alignment, the case for staying threaded, the case for tokio, migration cost, trigger conditions, and a five-step adoption sequence.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
) Defines the empirical benchmark harness, workloads (100 / 1k / 10k concurrent clients), metrics, soft-limit triggers, comparison oracle against upstream rsync 3.4.1, and the decision criteria that gate the async listener migration tracked under #1935. Frames the active-counter fix from the parent audit (#1673, PR #3705) as a strict precondition. Tracking: oc-rsync task #1933.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…#3891) Slim runnable plan complementing the broader benchmark plan. Specifies the minimum harness needed to land first measured numbers on Linux loopback at 100/1K/10K concurrent rsync:// pulls so #1934 RFC and #1935 async-listener work can compare sync vs async paths against quantified sync-baseline ttfb, completion p99, peak RSS, and thread count.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
#1933) (#4182) * chore(bench): stress harness for thread-per-connection scaling Adds an integration-test-shaped stress benchmark that drives 100, 1000, and 10000 concurrent TCP clients against the daemon listener, capturing wall time, ECONNREFUSED / EMFILE counts, and peak RSS via getrusage. All three scenarios are marked #[ignore]; the 10k case is unix-only and self-skips when RLIMIT_NOFILE cannot accommodate the request. The harness exists to provide evidence for whether an async listener (tracked separately) would be a meaningful change to the current std::thread::spawn-per-connection model. * style(daemon): apply rustfmt to connection_scaling_stress * fix(daemon): convert ru_maxrss via i64::from for cross-arch portability * fix(daemon): use ru_maxrss directly without conversion The conversion was redundant on all targets we build on: c_long is i64 on 64-bit Linux/macOS (no conversion needed) and i32 on 32-bit Linux (the subsequent 'as u64' sign-extends to i64 first, then saturating_mul operates on u64). Removing the conversion silences both clippy::unnecessary_cast and clippy::useless_conversion without losing overflow safety.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
Add a focused evaluation of async runtime options for the daemon accept loop. Compares tokio against async-std and the existing thread-per-connection model, and records the decision to adopt tokio with the rt-multi-thread flavour under the existing async feature gate. The doc complements the implementation plan in #1935 and the benchmark plan in #1933 rather than restating either. It covers maintenance posture, feature parity, ecosystem alignment, the case for staying threaded, the case for tokio, migration cost, trigger conditions, and a five-step adoption sequence.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
#1933) (#4182) * chore(bench): stress harness for thread-per-connection scaling Adds an integration-test-shaped stress benchmark that drives 100, 1000, and 10000 concurrent TCP clients against the daemon listener, capturing wall time, ECONNREFUSED / EMFILE counts, and peak RSS via getrusage. All three scenarios are marked #[ignore]; the 10k case is unix-only and self-skips when RLIMIT_NOFILE cannot accommodate the request. The harness exists to provide evidence for whether an async listener (tracked separately) would be a meaningful change to the current std::thread::spawn-per-connection model. * style(daemon): apply rustfmt to connection_scaling_stress * fix(daemon): convert ru_maxrss via i64::from for cross-arch portability * fix(daemon): use ru_maxrss directly without conversion The conversion was redundant on all targets we build on: c_long is i64 on 64-bit Linux/macOS (no conversion needed) and i32 on 32-bit Linux (the subsequent 'as u64' sign-extends to i64 first, then saturating_mul operates on u64). Removing the conversion silences both clippy::unnecessary_cast and clippy::useless_conversion without losing overflow safety.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
Add a focused evaluation of async runtime options for the daemon accept loop. Compares tokio against async-std and the existing thread-per-connection model, and records the decision to adopt tokio with the rt-multi-thread flavour under the existing async feature gate. The doc complements the implementation plan in #1935 and the benchmark plan in #1933 rather than restating either. It covers maintenance posture, feature parity, ecosystem alignment, the case for staying threaded, the case for tokio, migration cost, trigger conditions, and a five-step adoption sequence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
roll_many's bulk math by caching the iteration weight and reusing a scalar fallback helperu32so extremely large windows fall back to the scalar loopTesting
https://chatgpt.com/codex/tasks/task_e_6909b05a91288323aec39c2107cb63bc