io_uring

[randomizedcoder]

Gday @JackDoan ,

I hope you are doing well.

I noticed that Nebula is not yet using io_uring (yet).

( https://developers.redhat.com/articles/2023/04/12/why-you-should-use-iouring-network-io )

Io_uring should give a nice little speedup, but probably the larger impact will be reduced load on the system. From a functionality perspective it will be a noop, it'll just be faster and use less resources.

A few months back I added it to another project that does a lot of UDP, and it was great ( I'm actually carrying a fork, cos there was something that needed a tweak: https://github.com/randomizedcoder/giouring ):
https://github.com/randomizedcoder/gosrt/

Maybe you guys can have a look at io_uring please, and let me know if it sounds like a reasonable pull request I can create. Maybe the first step would be a small design doc, which should help us get aligned and (hopefully) make the PR easier to read.

Thanks in advance,
Dave

[JackDoan]

We've looked into io_uring before, but right now, we think it's higher impact to pursue performance via GSO/GRO-type offloads, on both the tun side and the UDP side. With the changes in #1704 (still a draft, but functional) I'm able to hit well over 10gbit on hardware that previously only managed ~2gbit. If you're trying to go fast, please take it for a spin!

Another reason is that our bottleneck isn't UDP, it's the tun device. Unfortunately, tun devices don't implement read_iter/write_iter, so they don't get the same benefits that sockets do from io_uring. :/