Skip to content

Experimental io_uring AIO implementation for disk IO#8992

Merged
cmcfarlen merged 17 commits into
apache:masterfrom
cmcfarlen:io_uring_disk_io
Oct 26, 2022
Merged

Experimental io_uring AIO implementation for disk IO#8992
cmcfarlen merged 17 commits into
apache:masterfrom
cmcfarlen:io_uring_disk_io

Conversation

@cmcfarlen

Copy link
Copy Markdown
Contributor

This PR adds support for using io_uring for AIO on Linux hosts.

For this implementation, each net thread and the main thread would have its own io_uring command queue for doing disk io. The io_uring queue processing is integrated into the net thread's epoll loop via eventfd. In this approach, there are no AIO specific threads and any request specifying AIO thread completion will be handled in the the net/main thread.

@bryancall bryancall added the Core label Aug 1, 2022
@bryancall bryancall added this to the 10.0.0 milestone Aug 1, 2022
@bryancall

Copy link
Copy Markdown
Contributor

[approve ci]

@maskit

maskit commented Aug 2, 2022

Copy link
Copy Markdown
Member

I wonder if we can also have a run-time setting (i.e. one for records.config) to enable/disable io_uring. It would be useful for testing/trouble shooting. And it would be also nice to be able to enable io_uring for disk IO and socket IO separately (e.g. 0: disabled, 1: just disk IO, 2: just socket IO, 3: the both).

@cmcfarlen cmcfarlen marked this pull request as ready for review August 30, 2022 19:55
@bryancall bryancall self-requested a review September 12, 2022 23:42
bryancall
bryancall previously approved these changes Oct 24, 2022
@cmcfarlen

Copy link
Copy Markdown
Contributor Author

[approve ci]

1 similar comment
@cmcfarlen

Copy link
Copy Markdown
Contributor Author

[approve ci]

@cmcfarlen cmcfarlen merged commit 2bd6ff4 into apache:master Oct 26, 2022
@cmcfarlen cmcfarlen deleted the io_uring_disk_io branch October 26, 2022 15:05
SolidWallOfCode pushed a commit to SolidWallOfCode/trafficserver that referenced this pull request Nov 15, 2022
* add io_uring feature to build

* link liburing library. Start adding implementation

* rename to DiskHandler to match native aio

* better way to integrate with net

* wip

* add diskhandler to main thread, link sqe ops for aio ops with multiple operations

* cleanup

* build should work without io_uring configured

* add config

* Finish config, attach parent wq, configure bounded/unbounded queues

* restore accidental line removal

* remove unused include

* set io opcode and increment stats

* Forward declare DiskHandler

* io_uring stats

* fix AIO_STAT_COUNT

* fix newline at end of file

Co-authored-by: Chris McFarlen <cmcfarlen@apple.com>
moonchen pushed a commit to moonchen/trafficserver that referenced this pull request Jun 21, 2026
Add ATS-INTEGRATION.md investigating how the prototype's coroutine layer maps
onto real ATS plumbing. Key finding: the merged disk-AIO io_uring work (apache#8992)
already gives every ET_NET thread a thread_local IOUringContext that is submitted
and drained inside NetHandler::waitForActivity via an eventfd bridge. So there is
no new reactor or thread to build:

  - Seam 1 (backend)  == IOUringContext (local_context(), next_sqe, submit, service)
  - Seam 2 (resume)   == IOUringContext::service() -> IOUringCompletionHandler::
                         handle_complete(cqe), called in waitForActivity on the
                         owning EThread (affinity is free; runs under the NH mutex)

Includes a concrete UringOp awaitable that IS an IOUringCompletionHandler and
resumes a coroutine from the existing service() dispatch, replacing net-iouring's
hand-written IOUringReader/IOUringWriter + ops_in_flight with one awaitable per
suspend. Notes the caveats: sockets are still epoll on master (add socket ops to
the same context, as the net-iouring branch did), close needs prep_cancel, and
submit is already batched per loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013RVTMqDkJfzkeKY6u7kVEE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants