-
Notifications
You must be signed in to change notification settings - Fork 399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQPOLL: io_uring_submit return value greater than what's newly submitted #88
Comments
Posting the full reproducer would be great, as it saves me time! |
I doubled the ring size (to 2N) and it works without the problem. |
I have also observed this problem. I'm using the code at tag liburing-0.7, Ubuntu 18.04 VM and 5.6.0-rc4 kernel. I haven't created a minimal reproducer since I just now figured out this was the issue and was looking for other reports of it |
Please do create one, it's the easiest way to discuss the issue as well. Thanks! |
I've created an minimal example with some error reporting here. I get that ideally you wouldn't submit sqes one at a time, but this is just to show the issue. Output that I get when running this is something like the following:
If it makes any difference, this is running on top of an xfs file system. |
SQPOLL is basically a parallel thread that taking out all SQEs |
Moreover, it can consume SQEs even if you don't do any syscalls (by polling). |
Hmm, it looks like I've got the example wrong, my apologies. I'll try to run it. |
yes, I understand that it will consume SQEs without a syscall as long as the kernel thread hasn't been put to sleep (in which case a syscall to wake it will be required, though I believe liburing handles this?). Whether or not |
I've got the same results for the latest kernel. Nice reproducer! syscall io_uring_enter() doesn't do much but returns @to_submit, which is In other words, if submission succeded, for SQPOLL the return value shows |
Makes sense, thanks for the explanation. I guess in this case it makes more sense for applications to use the result of io_uring_submit() just to make sure that it didn't fail completely and do their own tracking of how many requests have been created and submitted so they can check for completions. This is because, and correct me if I'm wrong, there's no way for and application to ask liburing for the number of in-flight operations that will eventually appear in the completion queue (and thus how many operations to wait on if they want to wait for all submitted I/O to complete) |
Indeed, and it's really not possible to make this fully accurate. For SQPOLL, you'll have to retain this count yourself so you know how much is pending. There is no way to ask for the number in-flight, |
A strange behavior observed in recent io_uring usage.
When not using SQPOLL, io_uring_submit returns correct numbers of sqes recently filled.
WIth SQPOLL, the first submit seems to be correct. But in the future, when wait_cqe(), seen(), and submit() are mixed, submit() may return a greater number, such as 2, or 3 when submitting one sqe. But when waiting cqes, the numbers of received cqes (number of returned wait()s) matches what's expected, which is smaller than then sum of the submit()'s return values.
It seems the test/read-write.c also checks for number of submissions. But I have no idea what's wrong in my code.
The code used to work correctly but started to have this issue now. It's been a few month since that same code was last run. Not sure if it's a bug in my code or in uring.
The pseudo code here: If necessary I can post the (length) full code that reproduces the error. (Linux 5.5.6, liburing #e42430c)
The text was updated successfully, but these errors were encountered: