Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about how liburing works #512

Closed
tickox opened this issue Jan 24, 2022 · 2 comments
Closed

Question about how liburing works #512

tickox opened this issue Jan 24, 2022 · 2 comments

Comments

@tickox
Copy link

tickox commented Jan 24, 2022

Hello,

My questions may have been asked before but I can't find an answer that suits me.

1)
If I create 1 ring on 1 thread, how many cores are used ? 1 or 2 ?
If I create a new ring on another thread, do I use 2 or 4 cores ?
(to be sure of the number of cores I will have left without having context switch)
If I understood correctly, there will be 2 cores used by ring (one for me and the other for the kernel).

2)
A thread waits via io_uring_wait_cqe on a ring.
A second thread requests a sqe and publishes it on the same ring as thread 1.

Is this a bug risk? (only thread 1 gets the cqe but one or more other threads can publish sqe)
If yes, what is the approach often used for this kind of scenario?

3)
Using blocking/non-blocking sockets changes something or not?
I assume that you should use non-blocking sockets.

Thanks

@isilence
Copy link
Collaborator

1) If I create 1 ring on 1 thread, how many cores are used ? 1 or 2 ? If I create a new ring on another thread, do I use 2 or 4 cores ? (to be sure of the number of cores I will have left without having context switch) If I understood correctly, there will be 2 cores used by ring (one for me and the other for the kernel).

In case no SQPOLL is specified, and there is a good chance you don't need it, there will be no additional threads. IOW, all io_uring syscalls will work in a context of the task it was called from.

If you use SQPOLL, it'll create one additional in-kernel thread doing polling and eating CPU per ring. However, you can share SQPOLL backend between rings (see IORING_SETUP_ATTACH_WQ), in that case you will get only 1 additional thread per N rings.

2) A thread waits via io_uring_wait_cqe on a ring. A second thread requests a sqe and publishes it on the same ring as thread 1.

Is this a bug risk? (only thread 1 gets the cqe but one or more other threads can publish sqe) If yes, what is the approach often used for this kind of scenario?

From the io_uring perspective you can safely have one thread doing completions and one submissions. Note, there was a catch with older kernels using any flavour of *wait_cqe_timeout(), see IORING_FEAT_EXT_ARG.

3) Using blocking/non-blocking sockets changes something or not? I assume that you should use non-blocking sockets.

It does, if there is no data it's not going to wait but would just return -EAGAIN. That behaviour may be wanted in some cases, e.g. if you do polling (including with poll requests).

@tickox
Copy link
Author

tickox commented Jan 24, 2022

Thanks you !

@tickox tickox closed this as completed Jan 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants