-
Notifications
You must be signed in to change notification settings - Fork 18.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
integration-cli: add more debugging for TestSwarmClusterRotateUnlockKey #39885
Merged
tiborvass
merged 1 commit into
moby:master
from
thaJeztah:debug_flaky_TestSwarmClusterRotateUnlockKey
Sep 19, 2019
Merged
integration-cli: add more debugging for TestSwarmClusterRotateUnlockKey #39885
tiborvass
merged 1 commit into
moby:master
from
thaJeztah:debug_flaky_TestSwarmClusterRotateUnlockKey
Sep 19, 2019
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Logs from that failing test; |
Failure on RS1 is #39857
|
bbef0e8
to
d3ab0ee
Compare
rebased; @cpuguy83 @tiborvass ptal |
tiborvass
approved these changes
Sep 13, 2019
This test was updated in b79adac, but is still flaky; ``` 20:24:13 FAIL: docker_cli_swarm_test.go:1333: DockerSwarmSuite.TestSwarmClusterRotateUnlockKey 20:24:13 20:24:13 Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/3/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey 20:24:13 [d6f95e679cb65] waiting for daemon to start 20:24:13 [d6f95e679cb65] waiting for daemon to start 20:24:13 [d6f95e679cb65] daemon started 20:24:13 20:24:13 Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/3/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey 20:24:13 [d204a02ba4780] waiting for daemon to start 20:24:13 [d204a02ba4780] waiting for daemon to start 20:24:13 [d204a02ba4780] daemon started 20:24:13 20:24:13 [d204a02ba4780] joining swarm manager [d6f95e679cb65]@0.0.0.0:2477, swarm listen addr 0.0.0.0:2478 20:24:13 Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/3/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey 20:24:13 [d873d6a842829] waiting for daemon to start 20:24:13 [d873d6a842829] waiting for daemon to start 20:24:13 [d873d6a842829] daemon started 20:24:13 20:24:13 [d873d6a842829] joining swarm manager [d6f95e679cb65]@0.0.0.0:2477, swarm listen addr 0.0.0.0:2479 20:24:13 [d204a02ba4780] Stopping daemon 20:24:13 [d204a02ba4780] exiting daemon 20:24:13 [d204a02ba4780] Daemon stopped 20:24:13 [d204a02ba4780] waiting for daemon to start 20:24:13 [d204a02ba4780] waiting for daemon to start 20:24:13 [d204a02ba4780] daemon started 20:24:13 20:24:13 [d873d6a842829] Stopping daemon 20:24:13 [d873d6a842829] exiting daemon 20:24:13 [d873d6a842829] Daemon stopped 20:24:13 [d873d6a842829] waiting for daemon to start 20:24:13 [d873d6a842829] waiting for daemon to start 20:24:13 [d873d6a842829] daemon started 20:24:13 20:24:13 docker_cli_swarm_test.go:1413: 20:24:13 c.Assert(err, checker.IsNil, check.Commentf("%s", outs)) 20:24:13 ... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc000934240), Stderr:[]uint8(nil)} ("exit status 1") 20:24:13 ... Error response from daemon: rpc error: code = Unknown desc = The swarm does not have a leader. It's possible that too few managers are online. Make sure more than half of the managers are online. 20:24:13 20:24:13 20:24:13 [d6f95e679cb65] Stopping daemon 20:24:13 [d6f95e679cb65] exiting daemon 20:24:13 [d6f95e679cb65] Daemon stopped 20:24:13 [d204a02ba4780] Stopping daemon 20:24:13 [d204a02ba4780] exiting daemon 20:24:13 [d204a02ba4780] Daemon stopped 20:24:13 [d873d6a842829] Stopping daemon 20:24:13 [d873d6a842829] exiting daemon 20:24:13 [d873d6a842829] Daemon stopped ``` The interesting bit there is that the retry loop should have a 3 second sleep before retrying, but looking at the failure above, the test started (and failed) within a second, which means that a different error / output was returned. This patch adds some additional debugging to that test to see if we can catch the reason this test is still flaky. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
d3ab0ee
to
78d137d
Compare
rebased |
tiborvass
approved these changes
Sep 19, 2019
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relates to #38885 Flaky test: DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
This test was updated in b79adac (#39616), but is still flaky #39883 (comment);
The interesting bit there is that the retry loop should have a 3 second sleep before retrying,
but looking at the failure above, the test started (and failed) within a second, which means that
a different error / output was returned.
This patch adds some additional debugging to that test to see if we can catch the reason
this test is still flaky.