Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GIT PULL] Support using a ring exclusively via registered index #664

Merged
merged 7 commits into from
Feb 21, 2023

Conversation

joshtriplett
Copy link
Contributor

@joshtriplett joshtriplett commented Sep 25, 2022

This provides the userspace corresponding to the kernel patch at
https://lore.kernel.org/io-uring/3cbedc531b633af4fe8632f7276aa843b5a54875.1664123680.git.josh@joshtriplett.org/T/
.

With this change, a caller of liburing can handle a ring internally without
leaving a file descriptor open that might disrupt its caller, which makes it
safer to retrofit existing libraries with strict compatibility requirements to
use io_uring.

Libraries with even more stringent requirements (e.g. never even transiently
having a file descriptor open) will need two additional pieces:

  • Adding a flag to io_uring_setup to set up the ring directly as a registered
    file descriptor, without ever putting it in the file descriptor table.
  • Supporting the initial mmap via a registered file descriptor, such as via
    an io_uring_register call.

git request-pull output:

The following changes since commit 65791341a5d7e79aa601c96be9df0e7ed4603e05:

  Merge branch 'test_print_counts' of https://github.com/ddiss/liburing (2022-09-23 18:58:59 -0600)

are available in the Git repository at:

  https://github.com/joshtriplett/liburing registered-ring-close

for you to fetch changes up to 5ba0c3c73f979d73e385e1d5a8acdac4c57a9ea8:

  Add example of closing a ring fd and using it via registered index (2022-09-25 21:54:47 +0100)

----------------------------------------------------------------
Josh Triplett (5):
      src/register.c: Call __sys_io_uring_register via a helper taking the ring
      Error on duplicate ring fd registration
      Support using IORING_REGISTER_USE_REGISTERED_RING
      Support closing the ring fd and using it exclusively via registered index
      Add example of closing a ring fd and using it via registered index

 examples/Makefile               |   1 +
 examples/io_uring-close-test.c  | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 man/io_uring_close_ring_fd.3    |  43 +++++++++++++++++++++++++++++++++++++++++++
 man/io_uring_register.2         |   8 ++++++++
 man/io_uring_setup.2            |   7 +++++++
 src/include/liburing.h          |   1 +
 src/include/liburing/io_uring.h |   6 +++++-
 src/liburing.map                |   1 +
 src/register.c                  | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------------------------
 src/setup.c                     |   3 ++-
 10 files changed, 250 insertions(+), 68 deletions(-)
 create mode 100644 examples/io_uring-close-test.c
 create mode 100644 man/io_uring_close_ring_fd.3

Click to show/hide pull request guidelines

Pull Request Guidelines

  1. To make everyone easily filter pull request from the email
    notification, use [GIT PULL] as a prefix in your PR title.
[GIT PULL] Your Pull Request Title
  1. Follow the commit message format rules below.
  2. Follow the Linux kernel coding style (see: https://github.com/torvalds/linux/blob/master/Documentation/process/coding-style.rst).

Commit message format rules:

  1. The first line is title (don't be more than 72 chars if possible).
  2. Then an empty line.
  3. Then a description (may be omitted for truly trivial changes).
  4. Then an empty line again (if it has a description).
  5. Then a Signed-off-by tag with your real name and email. For example:
Signed-off-by: Foo Bar <foo.bar@gmail.com>

The description should be word-wrapped at 72 chars. Some things should
not be word-wrapped. They may be some kind of quoted text - long
compiler error messages, oops reports, Link, etc. (things that have a
certain specific format).

Note that all of this goes in the commit message, not in the pull
request text. The pull request text should introduce what this pull
request does, and each commit message should explain the rationale for
why that particular change was made. The git tree is canonical source
of truth, not github.

Each patch should do one thing, and one thing only. If you find yourself
writing an explanation for why a patch is fixing multiple issues, that's
a good indication that the change should be split into separate patches.

If the commit is a fix for an issue, add a Fixes tag with the issue
URL.

Don't use GitHub anonymous email like this as the commit author:

123456789+username@users.noreply.github.com

Use a real email address!

Commit message example:

src/queue: don't flush SQ ring for new wait interface

If we have IORING_FEAT_EXT_ARG, then timeouts are done through the
syscall instead of by posting an internal timeout. This was done
to be both more efficient, but also to enable multi-threaded use
the wait side. If we touch the SQ state by flushing it, that isn't
safe without synchronization.

Fixes: https://github.com/axboe/liburing/issues/402
Signed-off-by: Jens Axboe <axboe@kernel.dk>

By submitting this pull request, I acknowledge that:

  1. I have followed the above pull request guidelines.
  2. I have the rights to submit this work under the same license.
  3. I agree to a Developer Certificate of Origin (see https://developercertificate.org for more information).

src/register.c Outdated Show resolved Hide resolved
src/include/liburing.h Show resolved Hide resolved
@axboe
Copy link
Owner

axboe commented Oct 20, 2022

Josh, this hasn't been forgotten. Let's pick this back up when 2.3 is released shortly, ditto on the kernel page as we're now outside of the merge window and can focus on 6.2 changes.

@ammarfaizi2
Copy link
Contributor

Josh, this hasn't been forgotten. Let's pick this back up when 2.3 is
released shortly, ditto on the kernel page as we're now outside of
the merge window and can focus on 6.2 changes.

Just a reminder. The 6.3 merge window will come soon. Any update on this?

@joshtriplett
Copy link
Contributor Author

Rebasing the kernel patch now.

@axboe
Copy link
Owner

axboe commented Feb 16, 2023

@joshtriplett Since the kernel patch is now staged for 6.3, can you update the liburing side too?

@axboe
Copy link
Owner

axboe commented Feb 16, 2023

Forgot to mention, it should also include a test verifying basic functionality and misuse.

…ring

In preparation for allowing register via a registered ring fd.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
This avoids overwriting and leaking a registered ring fd.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
@joshtriplett
Copy link
Contributor Author

Updated.

Looking into tests.

Copy link
Contributor

@ammarfaizi2 ammarfaizi2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we have a new rule. Since commit:

9e2890d ("build: add liburing-ffi")

Adding a new exported function should also be reflected in liburing-ffi.map.

man/io_uring_close_ring_fd.3 Outdated Show resolved Hide resolved
src/liburing.map Outdated Show resolved Hide resolved
@joshtriplett
Copy link
Contributor Author

Adding a new exported function should also be reflected in liburing-ffi.map.

Done.

@joshtriplett
Copy link
Contributor Author

@axboe Added a test. Let me know if the test covers what you'd like covered.

src/register.c Outdated Show resolved Hide resolved
@ammarfaizi2
Copy link
Contributor

Forgot to mention about the CHANGELOG file.

Add a note about this new API in the CHANGELOG file. The change can be
folded into the commit that adds the new function.

Use it if the kernel supports IORING_FEAT_REG_REG_RING and the user
registered the ring fd.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
…ndex

Add io_uring_close_fd to close the ring fd, and perform all subsequent
operations via registered index only.

Require the kernel to support IORING_FEAT_REG_REG_RING, to allow for
subsequent io_uring_register operations as well.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
Based on io_uring-test, modified to use io_uring_close_ring_fd.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
Make sure an io_uring_register operation works after
io_uring_close_ring_fd. Also test various error paths.

Tested on a kernel with IORING_REGISTER_USE_REGISTERED_RING support.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
Introduce the new internal flag INT_FLAG_REG_REG_RING, set when setting
INT_FLAG_REG_RING if the kernel supports IORING_FEAT_REG_REG_RING. This
allows all the register functions to just check INT_FLAG_REG_REG_RING.

Suggested-by: Dylan Yudaken <dylany@meta.com>
Signed-off-by: Josh Triplett <josh@joshtriplett.org>
@joshtriplett
Copy link
Contributor Author

@ammarfaizi2 Done.

@axboe axboe merged commit 2a0cdac into axboe:master Feb 21, 2023
@joshtriplett joshtriplett deleted the registered-ring-close branch February 23, 2023 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants