Restrict limiter test-only re-export#1860
Merged
Merged
Conversation
5 tasks
oferchen
added a commit
that referenced
this pull request
Apr 28, 2026
…ine (#3418) Adds two doc-only notes covering recurring questions: - crates/rsync_io/src/ssh/mod.rs: explains why SSH transfers cannot use the fast_io socket reader/writer paths. The data channel is the inherited stdio pipe pair from the spawned ssh child, not an AF_INET socket, so the io_uring socket fast path is unreachable. Mirrors upstream main.c:504 do_cmd(). Cross-references tasks #1859 (pipe-FD io_uring) and #1860 (splice-based zero-copy) and notes that daemon TCP and local-disk transfers still benefit from fast_io. - crates/transfer/src/generator/mod.rs: documents the sender-side INC_RECURSE state machine (Idle -> ScanDir -> SendChunk -> WaitAck -> NextDir -> Done) with upstream citations to flist.c:2192 send_file_list(), sender.c:199 send_files(), generator.c:2226 generate_files(), receiver.c:522 recv_files(), and compat.c:161 set_allow_inc_recurse(). Documents that the sender side is implemented but disabled in build_capability_string until interop is validated (task #1862); the 'i' capability is not advertised when oc-rsync is sender. Closes #1858, #1865.
4 tasks
oferchen
added a commit
that referenced
this pull request
Apr 29, 2026
* docs(transport): investigate splice()/vmsplice() for SSH stdio (#1860) Audit confirms upstream rsync 3.4.1 does not use splice. Identifies the file-to-pipe (sender) and pipe-to-file (receiver) edges as the only realistic targets for zero-copy on the SSH stdio path, since the parent does not own the SSH child's TCP socket. Documents the multiplex envelope constraint (4-byte header + 24-bit payload) and proposes a three-phase plan: payload splice on the sender, payload splice on the receiver, and vmsplice for the header ring. Risks cover vmsplice page lifetime, SPLICE_F_NONBLOCK semantics, EPIPE on closed peers, and interaction with --bwlimit. Adds docs/perf-roadmap.md as the index. * bench(fast_io): add splice_pipe benchmark skeleton (#1860) Provides the criterion baseline for the SSH-stdio splice investigation. The Linux body runs a real read+write_all loop from a temp file through a drained pipe at three sizes (64 KiB, 1 MiB, 16 MiB), establishing the comparison point for the future splice driver. The splice_baseline slot is a placeholder that records a constant for now; follow-up #1861 wires the real fast_io::splice_pipe driver into that slot. Non-Linux targets compile to a no-op group so cargo bench succeeds on macOS and Windows.
oferchen
added a commit
that referenced
this pull request
May 1, 2026
…ine (#3418) Adds two doc-only notes covering recurring questions: - crates/rsync_io/src/ssh/mod.rs: explains why SSH transfers cannot use the fast_io socket reader/writer paths. The data channel is the inherited stdio pipe pair from the spawned ssh child, not an AF_INET socket, so the io_uring socket fast path is unreachable. Mirrors upstream main.c:504 do_cmd(). Cross-references tasks #1859 (pipe-FD io_uring) and #1860 (splice-based zero-copy) and notes that daemon TCP and local-disk transfers still benefit from fast_io. - crates/transfer/src/generator/mod.rs: documents the sender-side INC_RECURSE state machine (Idle -> ScanDir -> SendChunk -> WaitAck -> NextDir -> Done) with upstream citations to flist.c:2192 send_file_list(), sender.c:199 send_files(), generator.c:2226 generate_files(), receiver.c:522 recv_files(), and compat.c:161 set_allow_inc_recurse(). Documents that the sender side is implemented but disabled in build_capability_string until interop is validated (task #1862); the 'i' capability is not advertised when oc-rsync is sender. Closes #1858, #1865.
oferchen
added a commit
that referenced
this pull request
May 1, 2026
* docs(transport): investigate splice()/vmsplice() for SSH stdio (#1860) Audit confirms upstream rsync 3.4.1 does not use splice. Identifies the file-to-pipe (sender) and pipe-to-file (receiver) edges as the only realistic targets for zero-copy on the SSH stdio path, since the parent does not own the SSH child's TCP socket. Documents the multiplex envelope constraint (4-byte header + 24-bit payload) and proposes a three-phase plan: payload splice on the sender, payload splice on the receiver, and vmsplice for the header ring. Risks cover vmsplice page lifetime, SPLICE_F_NONBLOCK semantics, EPIPE on closed peers, and interaction with --bwlimit. Adds docs/perf-roadmap.md as the index. * bench(fast_io): add splice_pipe benchmark skeleton (#1860) Provides the criterion baseline for the SSH-stdio splice investigation. The Linux body runs a real read+write_all loop from a temp file through a drained pipe at three sizes (64 KiB, 1 MiB, 16 MiB), establishing the comparison point for the future splice driver. The splice_baseline slot is a placeholder that records a constant for now; follow-up #1861 wires the real fast_io::splice_pipe driver into that slot. Non-Linux targets compile to a no-op group so cargo bench succeeds on macOS and Windows.
8 tasks
oferchen
added a commit
that referenced
this pull request
May 1, 2026
…nsport (#1938) (#3525) Formalizes the SSH stdio transport audit at docs/audits/ssh-socketpair-vs-pipes.md under tracker #1938, consolidating prior #1686 working notes (PR #3438), the io_uring boundary documented for #1858 (PR #3418), and the stderr socketpair shipped under #1689 (PR #3383). Restructures the document into the formal seven-section layout (summary, upstream reference, current implementation, trade-offs, decision matrix, recommendation, implementation notes) plus findings and references. Recommends keeping anonymous pipes for the SSH wire and not pursuing the socketpair migration: splice(2) and vmsplice(2) require a pipe end, the zero-copy plan in #1860 depends on that, and the unified-FD argument is subsumed by the async-transport refactor (#2068, #1655). Closes #1687 (prototype socketpair wire) as do-not-implement and #1902 (verify wire claim against rsync_io source) as verified. Citations were re-checked against worktree source: builder.rs:300-301 (Stdio::piped on the wire), aux_channel.rs:263-285 (UnixStream::pair for stderr, Unix only), connection.rs:30-39 (SshConnection struct), mod.rs:57-75 (io_uring boundary). Upstream evidence verified against target/interop/upstream-src/rsync-3.4.1/: pipe.c:48-97 (piped_child), util1.c:74-96 (fd_pair), main.c:504-663 (do_cmd dispatch), clientserver.c:116-148 (daemon TCP wire), socket.c:736-846 (sock_exec test escape).
oferchen
added a commit
that referenced
this pull request
May 5, 2026
…ine (#3418) Adds two doc-only notes covering recurring questions: - crates/rsync_io/src/ssh/mod.rs: explains why SSH transfers cannot use the fast_io socket reader/writer paths. The data channel is the inherited stdio pipe pair from the spawned ssh child, not an AF_INET socket, so the io_uring socket fast path is unreachable. Mirrors upstream main.c:504 do_cmd(). Cross-references tasks #1859 (pipe-FD io_uring) and #1860 (splice-based zero-copy) and notes that daemon TCP and local-disk transfers still benefit from fast_io. - crates/transfer/src/generator/mod.rs: documents the sender-side INC_RECURSE state machine (Idle -> ScanDir -> SendChunk -> WaitAck -> NextDir -> Done) with upstream citations to flist.c:2192 send_file_list(), sender.c:199 send_files(), generator.c:2226 generate_files(), receiver.c:522 recv_files(), and compat.c:161 set_allow_inc_recurse(). Documents that the sender side is implemented but disabled in build_capability_string until interop is validated (task #1862); the 'i' capability is not advertised when oc-rsync is sender. Closes #1858, #1865.
oferchen
added a commit
that referenced
this pull request
May 5, 2026
* docs(transport): investigate splice()/vmsplice() for SSH stdio (#1860) Audit confirms upstream rsync 3.4.1 does not use splice. Identifies the file-to-pipe (sender) and pipe-to-file (receiver) edges as the only realistic targets for zero-copy on the SSH stdio path, since the parent does not own the SSH child's TCP socket. Documents the multiplex envelope constraint (4-byte header + 24-bit payload) and proposes a three-phase plan: payload splice on the sender, payload splice on the receiver, and vmsplice for the header ring. Risks cover vmsplice page lifetime, SPLICE_F_NONBLOCK semantics, EPIPE on closed peers, and interaction with --bwlimit. Adds docs/perf-roadmap.md as the index. * bench(fast_io): add splice_pipe benchmark skeleton (#1860) Provides the criterion baseline for the SSH-stdio splice investigation. The Linux body runs a real read+write_all loop from a temp file through a drained pipe at three sizes (64 KiB, 1 MiB, 16 MiB), establishing the comparison point for the future splice driver. The splice_baseline slot is a placeholder that records a constant for now; follow-up #1861 wires the real fast_io::splice_pipe driver into that slot. Non-Linux targets compile to a no-op group so cargo bench succeeds on macOS and Windows.
oferchen
added a commit
that referenced
this pull request
May 5, 2026
…nsport (#1938) (#3525) Formalizes the SSH stdio transport audit at docs/audits/ssh-socketpair-vs-pipes.md under tracker #1938, consolidating prior #1686 working notes (PR #3438), the io_uring boundary documented for #1858 (PR #3418), and the stderr socketpair shipped under #1689 (PR #3383). Restructures the document into the formal seven-section layout (summary, upstream reference, current implementation, trade-offs, decision matrix, recommendation, implementation notes) plus findings and references. Recommends keeping anonymous pipes for the SSH wire and not pursuing the socketpair migration: splice(2) and vmsplice(2) require a pipe end, the zero-copy plan in #1860 depends on that, and the unified-FD argument is subsumed by the async-transport refactor (#2068, #1655). Closes #1687 (prototype socketpair wire) as do-not-implement and #1902 (verify wire claim against rsync_io source) as verified. Citations were re-checked against worktree source: builder.rs:300-301 (Stdio::piped on the wire), aux_channel.rs:263-285 (UnixStream::pair for stderr, Unix only), connection.rs:30-39 (SshConnection struct), mod.rs:57-75 (io_uring boundary). Upstream evidence verified against target/interop/upstream-src/rsync-3.4.1/: pipe.c:48-97 (piped_child), util1.c:74-96 (fd_pair), main.c:504-663 (do_cmd dispatch), clientserver.c:116-148 (daemon TCP wire), socket.c:736-846 (sock_exec test escape).
5 tasks
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
recorded_sleepsre-export in the bandwidth limiter to test-only builds to avoid unused import warningsTesting
https://chatgpt.com/codex/tasks/task_e_69057df7487c83239b34151d8a12f087