feat(fast_io): io_uring SEND_ZC network zero-copy primitive (#1832)#4301
Merged
Conversation
Adds `crates/fast_io/src/io_uring/send_zc.rs` exposing `try_send_zc(ring, fd, buf, user_data)` that submits an `IORING_OP_SEND_ZC` SQE, drains both the transfer CQE (`IORING_CQE_F_MORE`) and the notification CQE (`IORING_CQE_F_NOTIF`), and returns the byte count sent. Kernel support is probed via `IORING_REGISTER_PROBE` on first call and cached in a process-wide atomic; unsupported kernels surface `io::ErrorKind::Unsupported` so callers can fall back transparently. `IoUringSocketWriter::submit_send` now attempts SEND_ZC first when `IoUringConfig::allow_send_zc()` is true and the payload is at least 16 KiB (per the workload-B regression guard in `docs/design/iouring-send-zc.md` section 5), falling back to the existing `IORING_OP_SEND` batch path on `Unsupported`. A matching stub module on non-Linux returns `Unsupported` so cross-platform callers compile without `cfg`-gating. Unit tests cover the classify-CQE state machine, the empty-buffer rejection, and a 64 KiB loopback round-trip that succeeds only once the notification CQE has been observed.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…4301) Adds `crates/fast_io/src/io_uring/send_zc.rs` exposing `try_send_zc(ring, fd, buf, user_data)` that submits an `IORING_OP_SEND_ZC` SQE, drains both the transfer CQE (`IORING_CQE_F_MORE`) and the notification CQE (`IORING_CQE_F_NOTIF`), and returns the byte count sent. Kernel support is probed via `IORING_REGISTER_PROBE` on first call and cached in a process-wide atomic; unsupported kernels surface `io::ErrorKind::Unsupported` so callers can fall back transparently. `IoUringSocketWriter::submit_send` now attempts SEND_ZC first when `IoUringConfig::allow_send_zc()` is true and the payload is at least 16 KiB (per the workload-B regression guard in `docs/design/iouring-send-zc.md` section 5), falling back to the existing `IORING_OP_SEND` batch path on `Unsupported`. A matching stub module on non-Linux returns `Unsupported` so cross-platform callers compile without `cfg`-gating. Unit tests cover the classify-CQE state machine, the empty-buffer rejection, and a 64 KiB loopback round-trip that succeeds only once the notification CQE has been observed.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
…4301) Adds `crates/fast_io/src/io_uring/send_zc.rs` exposing `try_send_zc(ring, fd, buf, user_data)` that submits an `IORING_OP_SEND_ZC` SQE, drains both the transfer CQE (`IORING_CQE_F_MORE`) and the notification CQE (`IORING_CQE_F_NOTIF`), and returns the byte count sent. Kernel support is probed via `IORING_REGISTER_PROBE` on first call and cached in a process-wide atomic; unsupported kernels surface `io::ErrorKind::Unsupported` so callers can fall back transparently. `IoUringSocketWriter::submit_send` now attempts SEND_ZC first when `IoUringConfig::allow_send_zc()` is true and the payload is at least 16 KiB (per the workload-B regression guard in `docs/design/iouring-send-zc.md` section 5), falling back to the existing `IORING_OP_SEND` batch path on `Unsupported`. A matching stub module on non-Linux returns `Unsupported` so cross-platform callers compile without `cfg`-gating. Unit tests cover the classify-CQE state machine, the empty-buffer rejection, and a 64 KiB loopback round-trip that succeeds only once the notification CQE has been observed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
crates/fast_io/src/io_uring/send_zc.rsexposingtry_send_zc(ring, fd, buf, user_data)that submits anIORING_OP_SEND_ZCSQE and drains both the transfer CQE (IORING_CQE_F_MORE) and the notification CQE (IORING_CQE_F_NOTIF) before returning. Kernel support is probed viaIORING_REGISTER_PROBEon first call and cached in a process-wide atomic; unsupported kernels surfaceio::ErrorKind::Unsupported.IoUringSocketWriter::submit_sendto attempt SEND_ZC first whenIoUringConfig::allow_send_zc()is true and the payload is at least 16 KiB (workload-B regression guard fromdocs/design/iouring-send-zc.mdsection 5), falling back to the existingIORING_OP_SENDbatch path onUnsupported.cfg-gating; updates the design doc to reflect the landed primitive.Test plan
cargo fmt --allsend_zc::tests::*on the Linux runners (classify-CQE state machine, empty-buffer rejection, 64 KiB loopback round-trip)