What is the correct way to handle cancellation of in-flight requests?

[ioquatix]

Let's say I have several IO operations in flight A & B. Let's say they are all ready to read.

My CQ contains A and B and I iterate over CQEs using io_uring_for_each_cqe.

On processing A, let's say I cancel operation B and B's user_data becomes invalid.

On processing B, will it always be marked as cancelled, or is are there some cases where the CQ would not be updated in time/at all?

[axboe]

The way you are describing it, or at least the way I'm reading it, both A and B already completed. If they are in the CQ ring, that must be the case. Which means that if you issue a cancelation for B while processing A, then that cancelation will return -ENOENT as the request is no longer active/pending.

[ioquatix]

both A and B already completed

Yes, I think that's the worst case for cancellation handling. Can you think of any other odd scenarios?

that cancelation will return -ENOENT as the request is no longer active/pending.

That will be the result of the CQE for the cancellation request, correct?

[axboe]

Can you think of any other odd scenarios?

Not sure I follow, can you be specific? Not sure what "odd scenarios" means here. There are really 3 cases for cancelation:

1. Request was found and canceled
2. Request was found but in progress. I'll finish soon, maybe canceled, maybe not.
3. Request was not found

that cancelation will return -ENOENT as the request is no longer active/pending.
That will be the result of the CQE for the cancellation request, correct?

Yes, this is case 3 above - the request wasn't found since it's already posted a CQE and is long gone, so the cancelation request would post a CQE with -ENOENT.

[noteflakes]

1. Request was found and canceled
2. Request was found but in progress. I'll finish soon, maybe canceled, maybe not.
3. Request was not found

Just to make sure I understand correctly, case 3 will happen if the request has already finished, right? That means a CQE was already posted by the kernel and will have been handled by the app. Cases 1 and 2 will generate a CQE with either a -ECANCELED or a successful result. So in all three cases, a CQE will be received, which lets you know io_uring is definitely done with the request.

[axboe]

Right, it has either already posted a CQE, or is in the process of doing so. In that sense, it's no different than case 2 where you can also expect a successful completion if that request is racing with cancelation.

[ioquatix]

For clarification, this behaviour is better defined than epoll and kqueue IIUC. In other words, this behaviour helps build more efficient interfaces with io_uring, since we can avoid book keeping that is required with epoll and kqueue.