Real-time protocols and datagrams

[bschwind]

I see this excerpt in the docs:

We often see users reaching for the QUIC datagram extension when implementing real-time protocols. Doing this is in most cases misguided. QUIC datagram sending still interacts with QUIC’s congestion controller and thus are also acknowledged. Implementing traditional protocols on top of QUIC datagrams might thus not perform the way they were designed to. Instead, it’s often better to use lots of streams that are then stopped, reset or prioritized.

A real-world example is the media over QUIC protocol (MoQ in short): MoQ is used to transfer live video frames. It uses one QUIC stream for each frame (QUIC streams are cheap to create)!

Which API should I reach for when I want to send some larger payload that is split into chunks? The order of the chunks doesn't matter, and some packet loss is expected/acceptable, so my intuition says a QUIC stream isn't quite the right thing to use because that enforces order and retransmission, where neither is necessary. Should I just use the datagram API? The docs I quoted above are a bit off-putting because I don't know why reaching for it would be misguided.

[matheus23]

Short answer: Use uni-directional streams and cancel them when they're outdated and use stream priority to prioritize the most recent stream frames. See also https://moq.dev/blog/never-use-datagrams/

Longer answer: Nobody actually wants "unreliable" delivery. It's more likely that you have an application where more recent data makes older data obsolete. In that case, you'd cancel the older data once the more recent one is submitted. Or your data becomes obsolete after X milliseconds. In that case, you'd cancel streams after X milliseconds. Or you don't care about order/want the most recently submitted data to go through first. In that case, you'd add priorities to streams to ensure the most recently submitted streams have the highest priority and will be sent first.
Using the datagram extension in QUIC is also not quite what you'd expect. E.g. some people think of it as "just encrypted UDP datagrams", but in actuality they are also acknowledged. This is required for them to work together with congestion control. But you might actually do better with application-specific acknowledgements, if you were building on UDP directly, but well at that point you're building your own application-tailored QUIC. IMO you're almost always better served using QUIC's streams.

---

Which API should I reach for when I want to send some larger payload that is split into chunks? The order of the chunks doesn't matter

Keep in mind that QUIC streams can also be received out of order.

[bschwind]

E.g. some people think of it as "just encrypted UDP datagrams", but in actuality they are also acknowledged. This is required for them to work together with congestion control.

I'll admit that I thought of them this way, but from the docs I linked I was also aware they are acknowledged. But what implications does that acknowledgement really have? I would guess it just adds a bit of extra packet overhead, or perhaps some period acks if no other data is sent. Does acknowledgement have any other impacts? Or is the gist: "don't bother with the datagram API because it doesn't buy you anything over streams, since both trigger acknowledgement packets" ?

Keep in mind that QUIC streams can also be received out of order.

Oh this is cool, I hadn't yet discovered this API, this may be exactly what I want!

But for the sake of learning and properly understanding all of this, what exactly are the technical differences between an unordered unidirectional stream and something sent over QUIC datagrams? The receiver in both cases will send acknowledgement packets, and the stream will obviously re-transmit until all packets have been acknowledged. Is there anything else?

For the record, I'm currently sending compressed video frames in chunks that are generated from a fountain code, where it makes zero sense to ever re-transmit a packet because you can just generate another chunk from the fountain code and that chunk is as good as the one that was lost.

Thanks for taking the time to answer, cheers!

[flub]

If coming from datagrams it is very unlikely that you want the unordered receive stream API. That API is interesting when you e.g. transfer a file in a string and need to write it to disk. By reading unordered you can write the received chunks immediately without having to wait for retransmits of lost packets, thus increasing the flow control window. So that's a little more efficient.

For datagrams you are more likely to want to map each datagram to an individual stream. But if you receive part of that datagram (say in your case a frame) then it's probably not yet useful.