balance: Implement a Peak-EWMA load metric #76
Conversation
Responsible for tracking responses past their future
Demonstrates how tower-balance can be used to track h2 streams.
Update the demo to support response tracking Also, randomize response latencies to be a more interesting simulation.
tower-balance/src/load/peak_ewma.rs
Outdated
|
|
||
| use Load; | ||
|
|
||
| pub struct PeakEWMA<S, T> { |
There was a problem hiding this comment.
NIT: I believe that correct Rust style would call this PeakEwma.
|
🙌 |
|
I've improved the demo somewhat (moved to new tokio, randomized latencies, constrained concurrency, used the Measure api in both p2c cases), which helps to illustrate the performance benefits of peak ewma ( |
In order to better-utilize the reactor, and to ensure that the load balancer does not account for time waiting for the client to read responses, we wrap the balancer in a buffer.
|
After improving the test, we see much more pronounced benefit: Previously, the load balancer accounted for time wiating for the SendRequests tasks to process each future. Now this is done in separate tasks (in a threadpool). |
| let clock = clock::Clock::new_with_now(Now(time.clone())); | ||
|
|
||
| let mut enter = enter().expect("enter"); | ||
| clock::with_default(&clock, &mut enter, |_| { |
There was a problem hiding this comment.
@carllerche this works fine! though, ergonomically, this feels like boilerplate. I think we could probably package up a default mock timer that handles this.
This requires that implementers implement `Clone` so that strategy can be shared.
tower-balance/src/lib.rs
Outdated
| @@ -8,6 +8,7 @@ extern crate indexmap; | |||
| #[cfg(test)] | |||
| extern crate quickcheck; | |||
| extern crate rand; | |||
| extern crate tokio; | |||
There was a problem hiding this comment.
This should not depend on Tokio proper. It looks like it needs tokio-timer for clock.
tower-balance/src/load/peak_ewma.rs
Outdated
| // ===== impl PeakEwma ===== | ||
|
|
||
| impl<D: Discover> WithPeakEwma<D, NoInstrument> { | ||
| pub fn new(discover: D, decay: Duration) -> Self { |
There was a problem hiding this comment.
Is it going to be common to create an instance with NoInstrument?
There was a problem hiding this comment.
Probably not enough to warrant the split
olix0r
changed the title
Cargo.toml
Outdated
| @@ -21,3 +21,8 @@ log = "0.4.1" | |||
| env_logger = { version = "0.5.3", default-features = false } | |||
| tokio-timer = "0.1" | |||
| futures-cpupool = "0.1" | |||
|
|
|||
| [patch.crates-io] | |||
| tokio = { git = "https://github.com/tokio-rs/tokio", branch = "timer-abstract-now" } | |||
There was a problem hiding this comment.
should probably block merging this until this is released
There was a problem hiding this comment.
I'm trying to get it merged, but now CI is hanging!
|
Approved, but yeah. We are waiting for the upcoming tokio release (whenever CI decides to cooperate). |
The balancer provides an implementation of two load balancing strategies: RoundRobin and
P2C+LeastLoaded. The round-robin strategy is extremely simplistic and not sufficient for
most production systems. P2C+LL is a substantial improvement, but relies exclusively on
instantaneous information.
This change introduces P2C+PeakEWMA strategy. P2C+PE improves over P2C+LL by maintaining
an exponentially-weighted moving average of response latencies for each endpoint so that
the recent history directly factors into load balancing decisions. This technique was
pioneered by Finagle for use at Twitter. Finagle's P2C+PE implementation was
referenced heavily while developing this.
The provided demo can be used to illustrate the differences between load balacing
strategies. For example: