Add an examples that demonstrates p2c rr behavior #39
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The new _demo_ example sends a million simulated requests through each load balancer configuration and records theobserved latency distributions. Furthermore, this fixes a critcal bug in Balancer, where we did not properly iterate through not-ready nodes.
|
FWIW, I'm pretty sure that the bug you mentioned above also causes some test failures in |
|
Note that the panics I mentioned in #39 (comment) both occurred at I vote we try and fast-track getting this merged --- I'd like to test Conduit against this branch, but changing a |
|
Update: this appears to fix the panic I mentioned, but the same tests now appear to hang: It's unclear whether or not that's a bug in EDIT: doing some sleuthing and it looks like a Conduit bug, but this still remains to be determined. |
carllerche
reviewed
Jan 25, 2018
tower-balance/src/lib.rs
Outdated
| // Iterate through the not-ready endpoints from right to left to prevent removals | ||
| // from reordering services in a way that could prevent a service from being polled. | ||
| for idx in self.not_ready.len()-1..0 { | ||
| for offset in 1..n { |
There was a problem hiding this comment.
This can probably be....
for idx in (1..n).iter().rev
carllerche
approved these changes
Jan 25, 2018
hawkw
reviewed
Jan 25, 2018
tower-balance/src/lib.rs
Outdated
| for offset in 1..n { | ||
| let idx = n - offset; | ||
|
|
||
| for idx in (0..n-1).rev() { |
There was a problem hiding this comment.
@olix0r I don't think this change is correct --- I stopped seeing the integer overflow in Conduit tests when building with this branch as of commit fb08434, but after pointing back at master as of 777888d, I'm getting the overflow again:
running 3 tests
test outbound_times_out ... ignored
thread 'support proxy' panicked at 'thread 'attempt to subtract with overflowsupport proxy', ' panicked at '/Users/eliza/.cargo/git/checkouts/tower-b098c32cf5a1bcca/777888d/tower-balance/src/lib.rsattempt to subtract with overflow:', 154/Users/eliza/.cargo/git/checkouts/tower-b098c32cf5a1bcca/777888d/tower-balance/src/lib.rs::24154
:note: Run with `RUST_BACKTRACE=1` for a backtrace.
24
test outbound_reconnects_if_controller_stream_ends ... FAILED
test outbound_asks_controller_api ... FAILED
failures:
---- outbound_reconnects_if_controller_stream_ends stdout ----
thread 'outbound_reconnects_if_controller_stream_ends' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Inner(Error { kind: Io(Error { repr: Kind(BrokenPipe) }) }) }', src/libcore/result.rs:906:4
---- outbound_asks_controller_api stdout ----
thread 'outbound_asks_controller_api' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Inner(Error { kind: Io(Error { repr: Kind(BrokenPipe) }) }) }', src/libcore/result.rs:906:4
failures:
outbound_asks_controller_api
outbound_reconnects_if_controller_stream_ends
test result: FAILED. 0 passed; 2 failed; 1 ignored; 0 measured; 0 filtered out
hawkw
added a commit
that referenced
this pull request
Jan 25, 2018
An usize overflow can occur in `Balance::promote_to_ready` when `self.not_ready` has length 0. See my comment here: #39 (comment) Signed-off-by: Eliza Weisman <eliza@buoyant.io>
hawkw
added a commit
that referenced
this pull request
Jan 25, 2018
An usize overflow can occur in `Balance::promote_to_ready` when `self.not_ready` has length 0. See my comment here: #39 (comment) Signed-off-by: Eliza Weisman <eliza@buoyant.io>
hawkw
added a commit
that referenced
this pull request
Jan 26, 2018
The `..` syntax creates a _half-open_ range (see https://doc.rust-lang.org/std/ops/struct.Range.html), so all that messing about with `n-1` in #39 and #40 was never actually necessary. This actually fixes the Conduit test I mentioned in #39 (comment); it no longer hangs. Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The new demo example sends a million simulated requests through each load balancer configuration and records the observed latency distributions.
Furthermore, this fixes a critical bug in
Balancer, where we did not properly iterate through not-ready nodes.