fs: support io_uring with tokio::fs::read#7696
Merged
ADD-SP merged 7 commits intotokio-rs:masterfrom Dec 5, 2025
Merged
Conversation
Darksonn
reviewed
Oct 19, 2025
ADD-SP
reviewed
Oct 19, 2025
f4c97c2 to
7e73f17
Compare
ADD-SP
reviewed
Oct 19, 2025
Darksonn
reviewed
Oct 19, 2025
tokio/src/fs/read.rs
Outdated
| let mut offset = 0; | ||
|
|
||
| while size_read < size { | ||
| let left_to_read = (size - size_read) as u32; |
There was a problem hiding this comment.
Suggested change
| let left_to_read = (size - size_read) as u32; | |
| let left_to_read = u32::try_from(size - size_read).unwrap_or(u32::MAX); |
To properly support files bigger than 4GB.
Contributor
Author
There was a problem hiding this comment.
max read size at a time is u32::MAX, we read the rest in other next iterations
Contributor
Author
There was a problem hiding this comment.
In the future, if we know we're reading more than u32::MAX then we can batch 2 read requests to avoid extra syscalls
6237a4c to
636cfb8
Compare
martin-g
reviewed
Oct 28, 2025
martin-g
reviewed
Oct 28, 2025
ADD-SP
reviewed
Oct 28, 2025
Member
ADD-SP
left a comment
There was a problem hiding this comment.
This looks complicated, I will review it incrementally.
ADD-SP
reviewed
Oct 29, 2025
6e0abae to
8120700
Compare
mox692
reviewed
Nov 1, 2025
Member
mox692
left a comment
There was a problem hiding this comment.
I haven't checked all of the details in read_uring.rs, but left some comments I've noticed so far.
e6c6ce7 to
b9c3885
Compare
d219de6 to
a90b5c7
Compare
ADD-SP
reviewed
Dec 2, 2025
e1909d0 to
dfe83fe
Compare
Darksonn
reviewed
Dec 2, 2025
2c205e7 to
96c977e
Compare
Darksonn
approved these changes
Dec 4, 2025
ADD-SP
reviewed
Dec 5, 2025
martin-g
reviewed
Dec 5, 2025
martin-g
approved these changes
Dec 5, 2025
Reading 64 MB chunks at a time and keeping the kernel busy surpases
std::fs::read time with unoptimized io_uring one being 1.12% fast
Fix tests fail on allocation failure
Doc fix and fix double subtraction of offset
Fix typos and use assert the stopped task's first poll
Add comments and panic on failed u32 conversion
Check the EOF internally in the `op_read` function
Fix test with noop waker
995185b to
fb23b82
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
We slowly use io uring everywhere here I made the simple change of supporting
fs::readhowever it might not be as simple. Let me know if the unsafe I used is correct or not.We currently use the blocking
std::fs::MetaDatato obtain file size for buffer capacity and extend the length of the vector according to the bytes read in the CQE. This implementation sounds good on paper to me.Later we should implement an internal statx helper, in this PR or a seperate PR to make our uring implementation less painful to use. As this pr failed #7616
Lets put statx helper in different PR to avoid merging an inefficient read implementation given io uring is about being more efficient in file IO
Solution
Continue adopting io uring
strace on a
tokio::fs::readafter this change