feat(daemon): tokio async listener behind --features async-daemon (#1935)#4278
Merged
Conversation
) Adds the implementation skeleton for the hybrid daemon listener accepted in the runtime-choice ADR: a tokio multi-thread runtime drives TcpListener::accept while each connection is handed to the existing synchronous worker via spawn_blocking. Default builds remain tokio-free. - New `async-daemon` cargo feature gates the entire path. - `crates/daemon/src/async_listener.rs` exposes `run_hybrid_listener`, taking an `Arc<dyn Fn>` sync worker and a shutdown flag. Unit tests bind on an ephemeral port and confirm accept-and-dispatch plus prompt shutdown work end-to-end. - `daemon::run_async_daemon` wires the listener to the existing daemon config and signal-flag plumbing. The per-connection worker is a documented stub; production wiring to the sync session handler ships in the follow-up tracked under #1935 once the trigger conditions from `docs/design/daemon-async-runtime-choice.md` are met. - Tokio features extended with `rt-multi-thread` and `macros` so the new builder path compiles alongside the existing `async` feature.
2 tasks
oferchen
added a commit
that referenced
this pull request
May 17, 2026
… (#4285) Audits whether prerequisites for #1876 have landed and proposes splitting the work into reviewable subtasks (#1876-a through e). - #1874 (shared ring + POLL_ADD) shipped via PR #3553. - #1937 (session ring pool) shipped via PR #4275. - #1935 (hybrid tokio listener) shipped via PR #4278. - No io_uring task carries id #1875; the original dependency note most plausibly meant the session-pool and listener work above. Missing surface enumerated: pooled-ring constructors on the socket adapters, daemon plumbing for SessionRingPool, sync and async worker swaps, IORING_OP_LINK_TIMEOUT integration, and async-cancel discipline. Trigger conditions (kernel >= 5.6, io_uring feature, opt-in flag) and a five-step implementation plan included.
6 tasks
oferchen
added a commit
that referenced
this pull request
May 17, 2026
Scaffold the `IORING_OP_ASYNC_CANCEL` primitive now that the async daemon listener (#4278), session ring pool (#4275), and per-thread rings (#4288) have landed code paths that keep SQEs in-flight across abort decision points. Until now, every io_uring helper followed `submit + wait + drain` and never needed cancellation; the deferred design at `docs/design/iouring-async-cancel.md` documented the trigger condition this commit fulfils. - New module `crates/fast_io/src/io_uring/cancel.rs` (Linux + `io_uring` feature gated): - `cancel_by_user_data(&mut IoUring, u64) -> io::Result<CancelOutcome>` submits a single `IORING_OP_ASYNC_CANCEL` SQE matched against the target SQE's `user_data` tag and reports the kernel's terminal outcome. - `cancel_all_by_fd(&mut IoUring, RawFd) -> io::Result<usize>` uses `IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_ALL` (kernel 5.19+) to cancel every in-flight SQE that touches the given fd. - `enum CancelOutcome { Cancelled, NotFound, AlreadyComplete }` maps the three race outcomes (`0`, `-ENOENT`, `-EALREADY`) to typed variants so callers route into the correct buffer reclaim path. - Added `OpTag::Cancel` so cancel CQEs are demuxable from target CQEs sharing the same CQ; updated `SharedRing::reap` to flag a Cancel CQE as an InvalidData misuse rather than silently drop it. - Added cross-platform UAPI constants `IORING_OP_ASYNC_CANCEL`, `ASYNC_CANCEL_MIN_KERNEL`, `ASYNC_CANCEL_FD_MIN_KERNEL` in `io_uring_common`, mirrored in the non-Linux stub. - Stub module in `io_uring_stub.rs` returns `Unsupported` so cross-platform callers compile without `cfg` branching. - Re-exported the primitive from the `io_uring` module and `CancelOutcome` from the crate root. - Unit coverage: - `cancel_by_user_data_cancels_inflight_poll` submits a PollAdd on an empty pipe, cancels it, asserts `CancelOutcome::Cancelled`. - `cancel_by_user_data_reports_not_found_for_unknown_tag` asserts `-ENOENT` is folded into `NotFound`, not surfaced as an error. - `cancel_all_by_fd_cancels_inflight_polls` cancels two PollAdds bulk-by-fd (skips cleanly on kernels < 5.19). - `CancelOutcome::from_cqe_result` decode parity for each variant plus error surface for other errnos. - Stub mirrors assert `Unsupported` and constant parity.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
) (#4278) Adds the implementation skeleton for the hybrid daemon listener accepted in the runtime-choice ADR: a tokio multi-thread runtime drives TcpListener::accept while each connection is handed to the existing synchronous worker via spawn_blocking. Default builds remain tokio-free. - New `async-daemon` cargo feature gates the entire path. - `crates/daemon/src/async_listener.rs` exposes `run_hybrid_listener`, taking an `Arc<dyn Fn>` sync worker and a shutdown flag. Unit tests bind on an ephemeral port and confirm accept-and-dispatch plus prompt shutdown work end-to-end. - `daemon::run_async_daemon` wires the listener to the existing daemon config and signal-flag plumbing. The per-connection worker is a documented stub; production wiring to the sync session handler ships in the follow-up tracked under #1935 once the trigger conditions from `docs/design/daemon-async-runtime-choice.md` are met. - Tokio features extended with `rt-multi-thread` and `macros` so the new builder path compiles alongside the existing `async` feature.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
… (#4285) Audits whether prerequisites for #1876 have landed and proposes splitting the work into reviewable subtasks (#1876-a through e). - #1874 (shared ring + POLL_ADD) shipped via PR #3553. - #1937 (session ring pool) shipped via PR #4275. - #1935 (hybrid tokio listener) shipped via PR #4278. - No io_uring task carries id #1875; the original dependency note most plausibly meant the session-pool and listener work above. Missing surface enumerated: pooled-ring constructors on the socket adapters, daemon plumbing for SessionRingPool, sync and async worker swaps, IORING_OP_LINK_TIMEOUT integration, and async-cancel discipline. Trigger conditions (kernel >= 5.6, io_uring feature, opt-in flag) and a five-step implementation plan included.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
Scaffold the `IORING_OP_ASYNC_CANCEL` primitive now that the async daemon listener (#4278), session ring pool (#4275), and per-thread rings (#4288) have landed code paths that keep SQEs in-flight across abort decision points. Until now, every io_uring helper followed `submit + wait + drain` and never needed cancellation; the deferred design at `docs/design/iouring-async-cancel.md` documented the trigger condition this commit fulfils. - New module `crates/fast_io/src/io_uring/cancel.rs` (Linux + `io_uring` feature gated): - `cancel_by_user_data(&mut IoUring, u64) -> io::Result<CancelOutcome>` submits a single `IORING_OP_ASYNC_CANCEL` SQE matched against the target SQE's `user_data` tag and reports the kernel's terminal outcome. - `cancel_all_by_fd(&mut IoUring, RawFd) -> io::Result<usize>` uses `IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_ALL` (kernel 5.19+) to cancel every in-flight SQE that touches the given fd. - `enum CancelOutcome { Cancelled, NotFound, AlreadyComplete }` maps the three race outcomes (`0`, `-ENOENT`, `-EALREADY`) to typed variants so callers route into the correct buffer reclaim path. - Added `OpTag::Cancel` so cancel CQEs are demuxable from target CQEs sharing the same CQ; updated `SharedRing::reap` to flag a Cancel CQE as an InvalidData misuse rather than silently drop it. - Added cross-platform UAPI constants `IORING_OP_ASYNC_CANCEL`, `ASYNC_CANCEL_MIN_KERNEL`, `ASYNC_CANCEL_FD_MIN_KERNEL` in `io_uring_common`, mirrored in the non-Linux stub. - Stub module in `io_uring_stub.rs` returns `Unsupported` so cross-platform callers compile without `cfg` branching. - Re-exported the primitive from the `io_uring` module and `CancelOutcome` from the crate root. - Unit coverage: - `cancel_by_user_data_cancels_inflight_poll` submits a PollAdd on an empty pipe, cancels it, asserts `CancelOutcome::Cancelled`. - `cancel_by_user_data_reports_not_found_for_unknown_tag` asserts `-ENOENT` is folded into `NotFound`, not surfaced as an error. - `cancel_all_by_fd_cancels_inflight_polls` cancels two PollAdds bulk-by-fd (skips cleanly on kernels < 5.19). - `CancelOutcome::from_cqe_result` decode parity for each variant plus error surface for other errnos. - Stub mirrors assert `Unsupported` and constant parity.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
) (#4278) Adds the implementation skeleton for the hybrid daemon listener accepted in the runtime-choice ADR: a tokio multi-thread runtime drives TcpListener::accept while each connection is handed to the existing synchronous worker via spawn_blocking. Default builds remain tokio-free. - New `async-daemon` cargo feature gates the entire path. - `crates/daemon/src/async_listener.rs` exposes `run_hybrid_listener`, taking an `Arc<dyn Fn>` sync worker and a shutdown flag. Unit tests bind on an ephemeral port and confirm accept-and-dispatch plus prompt shutdown work end-to-end. - `daemon::run_async_daemon` wires the listener to the existing daemon config and signal-flag plumbing. The per-connection worker is a documented stub; production wiring to the sync session handler ships in the follow-up tracked under #1935 once the trigger conditions from `docs/design/daemon-async-runtime-choice.md` are met. - Tokio features extended with `rt-multi-thread` and `macros` so the new builder path compiles alongside the existing `async` feature.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
… (#4285) Audits whether prerequisites for #1876 have landed and proposes splitting the work into reviewable subtasks (#1876-a through e). - #1874 (shared ring + POLL_ADD) shipped via PR #3553. - #1937 (session ring pool) shipped via PR #4275. - #1935 (hybrid tokio listener) shipped via PR #4278. - No io_uring task carries id #1875; the original dependency note most plausibly meant the session-pool and listener work above. Missing surface enumerated: pooled-ring constructors on the socket adapters, daemon plumbing for SessionRingPool, sync and async worker swaps, IORING_OP_LINK_TIMEOUT integration, and async-cancel discipline. Trigger conditions (kernel >= 5.6, io_uring feature, opt-in flag) and a five-step implementation plan included.
oferchen
added a commit
that referenced
this pull request
May 18, 2026
Scaffold the `IORING_OP_ASYNC_CANCEL` primitive now that the async daemon listener (#4278), session ring pool (#4275), and per-thread rings (#4288) have landed code paths that keep SQEs in-flight across abort decision points. Until now, every io_uring helper followed `submit + wait + drain` and never needed cancellation; the deferred design at `docs/design/iouring-async-cancel.md` documented the trigger condition this commit fulfils. - New module `crates/fast_io/src/io_uring/cancel.rs` (Linux + `io_uring` feature gated): - `cancel_by_user_data(&mut IoUring, u64) -> io::Result<CancelOutcome>` submits a single `IORING_OP_ASYNC_CANCEL` SQE matched against the target SQE's `user_data` tag and reports the kernel's terminal outcome. - `cancel_all_by_fd(&mut IoUring, RawFd) -> io::Result<usize>` uses `IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_ALL` (kernel 5.19+) to cancel every in-flight SQE that touches the given fd. - `enum CancelOutcome { Cancelled, NotFound, AlreadyComplete }` maps the three race outcomes (`0`, `-ENOENT`, `-EALREADY`) to typed variants so callers route into the correct buffer reclaim path. - Added `OpTag::Cancel` so cancel CQEs are demuxable from target CQEs sharing the same CQ; updated `SharedRing::reap` to flag a Cancel CQE as an InvalidData misuse rather than silently drop it. - Added cross-platform UAPI constants `IORING_OP_ASYNC_CANCEL`, `ASYNC_CANCEL_MIN_KERNEL`, `ASYNC_CANCEL_FD_MIN_KERNEL` in `io_uring_common`, mirrored in the non-Linux stub. - Stub module in `io_uring_stub.rs` returns `Unsupported` so cross-platform callers compile without `cfg` branching. - Re-exported the primitive from the `io_uring` module and `CancelOutcome` from the crate root. - Unit coverage: - `cancel_by_user_data_cancels_inflight_poll` submits a PollAdd on an empty pipe, cancels it, asserts `CancelOutcome::Cancelled`. - `cancel_by_user_data_reports_not_found_for_unknown_tag` asserts `-ENOENT` is folded into `NotFound`, not surfaced as an error. - `cancel_all_by_fd_cancels_inflight_polls` cancels two PollAdds bulk-by-fd (skips cleanly on kernels < 5.19). - `CancelOutcome::from_cqe_result` decode parity for each variant plus error surface for other errnos. - Stub mirrors assert `Unsupported` and constant parity.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/design/daemon-async-runtime-choice.md: tokio multi-thread runtime drivestokio::net::TcpListener::accept, and each accepted connection is handed to the synchronous worker viatokio::task::spawn_blocking.async-daemoncargo feature, off by default. Default builds stay tokio-free and continue to dispatch through the existingserve_connectionsthread-per-connection loop.daemon::run_async_daemon, which reuses the existing config parsing and signal-flag plumbing. The per-connection worker is a documented stub; production wiring to the sync session handler is the follow-up tracked under Normalize quoted fallback keywords #1935 once the trigger conditions in the runtime-choice ADR are met (sustained >1k concurrent connections, blocking-pool starvation measurements, two release cycles of green async-daemon CI).Files
crates/daemon/Cargo.toml- declares theasync-daemonfeature and extends the optional tokio feature list withrt-multi-threadandmacros.crates/daemon/src/async_listener.rs- new module exposingrun_hybrid_listener(addr, worker_threads, shutdown, SyncWorker)plus accept loop, with unit tests for bind/accept/dispatch and prompt shutdown.crates/daemon/src/daemon.rs- addsrun_async_daemon, gated byasync-daemon, which feeds the parsed runtime options into the new listener and hooks into the existing signal flags.crates/daemon/src/lib.rs- re-exportsasync_listenerandrun_async_daemonbehind the feature flag.Test plan
--features async-daemon--features async-daemonexercises the two new unit tests inasync_listener::testsReferences
docs/design/daemon-async-runtime-choice.mddocs/design/daemon-tokio-async-listener-impl.mddocs/design/daemon-async-accept-sync-workers.md