perf: Use RTT variance instead of a fixed threshold

[bruceg]

Instead of using a fixed threshold, this calculates the standard deviation of the EWMA of the past RTT measurements, and uses a multiple of that as the threshold range.

[jszwedko]

🎉

I'll try this out with the tests we have so far.

[jszwedko]

Would you be able to explain a bit what this is doing?

[jszwedko]

(specifically the variance bit of the calculation)

[jszwedko]

I note that I see:

warning: method is never used: `new`
   --> src/sinks/util/auto_concurrency/controller.rs:266:5
    |
266 |     fn new(alpha: f64) -> Self {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(dead_code)]` on by default

warning: 1 warning emitted

When building.

I reran the tests (including the added "knee behavior" tests).

Some observations:

• It seems like it does better in the small variance case, but actually appears worse in the medium/large variance cases
• It seems to do better for the knee case with a higher slope

Before (424ea5e):

After (6670a0e):

I'm realizing it'd probably be useful to plot before/after on the same graphs (specifically the throughput) to make it easier to see differences. I opened vectordotdev/http_test_server#4 to that end.

[bruceg]

Since this doesn't seem to have helped any, at least with our simulations, and needs some work to be usable, I will close this for now and see what happens in real deployment situations.