docs: SSH stdio io_uring constraint and sender INC_RECURSE state machine by oferchen · Pull Request #3418 · oferchen/rsync

oferchen · 2026-04-28T20:05:32Z

Summary

Two doc-only additions covering questions that recur during reviews and onboarding. No code logic changes.

Task #1858 - SSH stdio io_uring limitation note

Extends the crates/rsync_io/src/ssh/mod.rs module rustdoc to explain:

The SSH data channel is the inherited stdio (stdin/stdout) of the spawned ssh child - a pipe pair, not an AF_INET/AF_INET6 socket.
The fast_io io_uring socket_reader / socket_writer paths require a socket FD, so they are unreachable for SSH regardless of kernel version or io_uring_policy.
Linux alternatives that exist for pipe FDs (IORING_OP_READ/IORING_OP_WRITE since 5.1, splice() / vmsplice() for zero-copy) are tracked separately as Gate alternatives registration and centralize tarball target metadata #1859 and Restrict limiter test-only re-export #1860.
Daemon-mode TCP and local-disk transfers DO benefit from fast_io.
Mirrors upstream rsync, which also performs blocking reads/writes on inherited stdio FDs (main.c:504 do_cmd()).

Task #1865 - Sender-side INC_RECURSE state machine doc

Extends the crates/transfer/src/generator/mod.rs module rustdoc with:

The state machine: Idle -> ScanDir -> SendChunk -> WaitAck -> NextDir -> Done.
Upstream citations (verified against target/interop/upstream-src/rsync-3.4.1/):
- flist.c:2192 send_file_list()
- flist.c send_extra_file_list()
- sender.c:199 send_files() (segment dispatch at top/bottom of loop)
- generator.c:2226 generate_files()
- receiver.c:522 recv_files()
- compat.c:161 set_allow_inc_recurse()
Status note: sender-side state machine and segment scheduler are implemented (IncrementalState / SegmentScheduler / PendingSegment), but 'i' is NOT advertised when oc-rsync is sender. build_capability_string is called with allow_inc_recurse = !is_sender until interop is validated (task Make Debian alternatives opt-in and clarify amd64 tarball distribution #1862). Pull transfers negotiate INC_RECURSE normally.

Closes #1858. Closes #1865.

Test plan

cargo fmt --all clean
CI: fmt+clippy
CI: nextest (stable)
CI: Windows / macOS / Linux musl
CI: rustdoc renders the new module docs without broken intra-doc links

Adds two doc-only notes covering recurring questions: - crates/rsync_io/src/ssh/mod.rs: explains why SSH transfers cannot use the fast_io socket reader/writer paths. The data channel is the inherited stdio pipe pair from the spawned ssh child, not an AF_INET socket, so the io_uring socket fast path is unreachable. Mirrors upstream main.c:504 do_cmd(). Cross-references tasks #1859 (pipe-FD io_uring) and #1860 (splice-based zero-copy) and notes that daemon TCP and local-disk transfers still benefit from fast_io. - crates/transfer/src/generator/mod.rs: documents the sender-side INC_RECURSE state machine (Idle -> ScanDir -> SendChunk -> WaitAck -> NextDir -> Done) with upstream citations to flist.c:2192 send_file_list(), sender.c:199 send_files(), generator.c:2226 generate_files(), receiver.c:522 recv_files(), and compat.c:161 set_allow_inc_recurse(). Documents that the sender side is implemented but disabled in build_capability_string until interop is validated (task #1862); the 'i' capability is not advertised when oc-rsync is sender. Closes #1858, #1865.

…ine (#3418) Adds two doc-only notes covering recurring questions: - crates/rsync_io/src/ssh/mod.rs: explains why SSH transfers cannot use the fast_io socket reader/writer paths. The data channel is the inherited stdio pipe pair from the spawned ssh child, not an AF_INET socket, so the io_uring socket fast path is unreachable. Mirrors upstream main.c:504 do_cmd(). Cross-references tasks #1859 (pipe-FD io_uring) and #1860 (splice-based zero-copy) and notes that daemon TCP and local-disk transfers still benefit from fast_io. - crates/transfer/src/generator/mod.rs: documents the sender-side INC_RECURSE state machine (Idle -> ScanDir -> SendChunk -> WaitAck -> NextDir -> Done) with upstream citations to flist.c:2192 send_file_list(), sender.c:199 send_files(), generator.c:2226 generate_files(), receiver.c:522 recv_files(), and compat.c:161 set_allow_inc_recurse(). Documents that the sender side is implemented but disabled in build_capability_string until interop is validated (task #1862); the 'i' capability is not advertised when oc-rsync is sender. Closes #1858, #1865.

…nsport (#1938) (#3525) Formalizes the SSH stdio transport audit at docs/audits/ssh-socketpair-vs-pipes.md under tracker #1938, consolidating prior #1686 working notes (PR #3438), the io_uring boundary documented for #1858 (PR #3418), and the stderr socketpair shipped under #1689 (PR #3383). Restructures the document into the formal seven-section layout (summary, upstream reference, current implementation, trade-offs, decision matrix, recommendation, implementation notes) plus findings and references. Recommends keeping anonymous pipes for the SSH wire and not pursuing the socketpair migration: splice(2) and vmsplice(2) require a pipe end, the zero-copy plan in #1860 depends on that, and the unified-FD argument is subsumed by the async-transport refactor (#2068, #1655). Closes #1687 (prototype socketpair wire) as do-not-implement and #1902 (verify wire claim against rsync_io source) as verified. Citations were re-checked against worktree source: builder.rs:300-301 (Stdio::piped on the wire), aux_channel.rs:263-285 (UnixStream::pair for stderr, Unix only), connection.rs:30-39 (SshConnection struct), mod.rs:57-75 (io_uring boundary). Upstream evidence verified against target/interop/upstream-src/rsync-3.4.1/: pipe.c:48-97 (piped_child), util1.c:74-96 (fd_pair), main.c:504-663 (do_cmd dispatch), clientserver.c:116-148 (daemon TCP wire), socket.c:736-846 (sock_exec test escape).

…ine (#3418) Adds two doc-only notes covering recurring questions: - crates/rsync_io/src/ssh/mod.rs: explains why SSH transfers cannot use the fast_io socket reader/writer paths. The data channel is the inherited stdio pipe pair from the spawned ssh child, not an AF_INET socket, so the io_uring socket fast path is unreachable. Mirrors upstream main.c:504 do_cmd(). Cross-references tasks #1859 (pipe-FD io_uring) and #1860 (splice-based zero-copy) and notes that daemon TCP and local-disk transfers still benefit from fast_io. - crates/transfer/src/generator/mod.rs: documents the sender-side INC_RECURSE state machine (Idle -> ScanDir -> SendChunk -> WaitAck -> NextDir -> Done) with upstream citations to flist.c:2192 send_file_list(), sender.c:199 send_files(), generator.c:2226 generate_files(), receiver.c:522 recv_files(), and compat.c:161 set_allow_inc_recurse(). Documents that the sender side is implemented but disabled in build_capability_string until interop is validated (task #1862); the 'i' capability is not advertised when oc-rsync is sender. Closes #1858, #1865.

…nsport (#1938) (#3525) Formalizes the SSH stdio transport audit at docs/audits/ssh-socketpair-vs-pipes.md under tracker #1938, consolidating prior #1686 working notes (PR #3438), the io_uring boundary documented for #1858 (PR #3418), and the stderr socketpair shipped under #1689 (PR #3383). Restructures the document into the formal seven-section layout (summary, upstream reference, current implementation, trade-offs, decision matrix, recommendation, implementation notes) plus findings and references. Recommends keeping anonymous pipes for the SSH wire and not pursuing the socketpair migration: splice(2) and vmsplice(2) require a pipe end, the zero-copy plan in #1860 depends on that, and the unified-FD argument is subsumed by the async-transport refactor (#2068, #1655). Closes #1687 (prototype socketpair wire) as do-not-implement and #1902 (verify wire claim against rsync_io source) as verified. Citations were re-checked against worktree source: builder.rs:300-301 (Stdio::piped on the wire), aux_channel.rs:263-285 (UnixStream::pair for stderr, Unix only), connection.rs:30-39 (SshConnection struct), mod.rs:57-75 (io_uring boundary). Upstream evidence verified against target/interop/upstream-src/rsync-3.4.1/: pipe.c:48-97 (piped_child), util1.c:74-96 (fd_pair), main.c:504-663 (do_cmd dispatch), clientserver.c:116-148 (daemon TCP wire), socket.c:736-846 (sock_exec test escape).

github-actions Bot added the documentation Improvements or additions to documentation label Apr 28, 2026

oferchen mentioned this pull request Apr 28, 2026

perf: set IOSQE_ASYNC on io_uring SEND submissions #3421

Closed

3 tasks

oferchen merged commit c4a84fc into master Apr 28, 2026
40 of 42 checks passed

oferchen deleted the docs/ssh-stdio-and-sender-inc-recurse branch April 28, 2026 22:48

oferchen mentioned this pull request May 1, 2026

docs(rsync_io): audit SSH socketpair vs anonymous pipes for stdio transport (#1938) #3525

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

docs: SSH stdio io_uring constraint and sender INC_RECURSE state machine#3418

docs: SSH stdio io_uring constraint and sender INC_RECURSE state machine#3418
oferchen merged 1 commit into
masterfrom
docs/ssh-stdio-and-sender-inc-recurse

oferchen commented Apr 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

oferchen commented Apr 28, 2026

Summary

Task #1858 - SSH stdio io_uring limitation note

Task #1865 - Sender-side INC_RECURSE state machine doc

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant