Add dynamic threadpool

[spacejam]

No description provided.

[spacejam]

• got rid of all of the dynamic backpressure stuff that would take time to tune
• made the channel unbuffered to give direct feedback to the work submission thread to spin up another blocking task when helpful. this reduces bufferbloat, which is also important to keep an eye on when we have so many implicit and explicit queues wiring everything together
• always wait 1 second before spinning down on the background work task

[ghost]

Looks so simple but effective, thanks! :)

[spacejam]

One potential issue is a thundering herd on the back-side of an IO burst, but I don't know if it's worth making the code harder to follow by adding an XOR rng inline to randomize the recv_timeout duration.

[ghost]

I'd be okay with adding a XOR rng inline and randomizing recv_timeout()! Sounds like a couple lines of code :)

Here, you can put this into src/utils.rs for generating random numbers:

/// Generates a random 32-bit integer.
pub fn random(n: u32) -> u32 {
    thread_local! {
        static RNG: Cell<Wrapping<u32>> = Cell::new(Wrapping(1406868647));
    }

    RNG.with(|rng| {
        // This is the 32-bit variant of Xorshift.
        //
        // Source: https://en.wikipedia.org/wiki/Xorshift
        let mut x = rng.get();
        x ^= x << 13;
        x ^= x >> 17;
        x ^= x << 5;
        rng.set(x);
        x.0
    })
}

I'm pretty sure we'll need to use a RNG in some other places soon so this would be handy to have!