Transmit directly from connection tasks

[Ralith]

This allows outgoing data to parallelize perfectly. Initial informal testing suggests a performance improvement for bulk data, likely due to reduced allocation and cross-task messaging. A larger performance benefit should be expected for endpoints hosting numerous connections.

I'd left the original userspace-multiplexed transmit strategy in place for a long time because I assumed the UDP socket had a mutex-guarded send buffer, so leaning into task-parallelism for sending would just lead to contention. This turns out to be complete nonsense, at least on Linux: outgoing UDP datagrams are buffered by the kernel with dynamically allocated memory, and the primitive NIC queuing operation is apparently scalable, as borne out by empirical testing. The following minimal test scales almost linearly with physical parallelism on Linux:

use std::{
    net::{SocketAddr, UdpSocket},
    thread,
    time::Instant,
};

fn main() {
    let sock = UdpSocket::bind("[::]:0").unwrap();
    let target: SocketAddr = "[::1]:1234".parse().unwrap();
    let start = Instant::now();
    const DATAGRAMS: usize = 1_000_000;
    thread::scope(|scope| {
        let threads = thread::available_parallelism().unwrap().get();
        dbg!(threads);
        let each = DATAGRAMS / threads;
        let sock = &sock;
        for _ in 0..threads {
            scope.spawn(move || {
                for _ in 0..each {
                    sock.send_to(b"hello, world", target).unwrap();
                }
            });
        }
    });
    let seconds = start.elapsed().as_secs_f32();
    println!("{} pps", DATAGRAMS as f32 / seconds);
}

It'd be interesting to see how this compares on other major platforms, but Linux's drastic improvement and the simplification of Quinn's internals are enough to satisfy me that we should go this way.

[Ralith]

Hmm, this needs a mechanism to broadcast wakeups when the UDP socket is backpressured...

edit: solved by duplicating (try_cloneing) the socket

[djc]

Looks cool -- initial round of feedback trying to make sense of it all.

[Ralith]

I've replaced the original try_clone_box mechanism with the runtime's built-in concurrent I/O support, which exists in both tokio and async-std. This required a somewhat fiddly extension of the runtime abstraction layer, because for both runtimes, concurrent I/O is only possible if you use async fn interfaces, probably due to self-reference somewhere in the critical futures. Luckily, the fiddly bits are isolated almost exclusively within runtime.rs, with both implementations and callers remaining straightforward. See discussion at tokio-rs/tokio#6226 for context.

[flub]

Hi, curious to try this out on various other platforms as well. How have you been running benchmarks to compare this?

I guess any perf loss due to losing sendmmsg is offset by the perf gain of not doing the userspace-multiplexing. sendmmsg does give some small benefit when you manage to fill the pipe to many peers in one syscall, but I'd have to spend more time figuring out how often quinn really manages to do that.

[Ralith]

I haven't benchmarked this very rigorously yet, since I don't have convenient access to a machine that's both running Linux (for our most optimized UDP backend) and not a laptop (subject to unpredictable throttling). We could use more data on that, if you're interested. I do plan to do some benchmarking on Windows, but we have neither sendmmsg nor GSO there just yet, so it might be a bit unfair to the status quo.

For a single connection, sendmmsg is strictly worse than GSO, and for multiple connections I think the near-linear speedup from parallelism will be a much bigger win in any case, so I'm not too concerned about regressions, though it's always possible we might miss something silly.

[Ralith]

Rebased.

[djc]

Looking good!