Skip to content
This repository has been archived by the owner on Oct 13, 2023. It is now read-only.

[19.03 backport] integration-cli: fix swarm tests flakiness #345

Merged
merged 3 commits into from
Sep 16, 2019

Conversation

thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Sep 1, 2019

based on top of / depends on #353 rebased

# (partial backport of) https://github.com/moby/moby/pull/39531 integration-cli: fix swarm tests flakiness
git cherry-pick -s -S -x 3df1095bbdc331d4effa5452d8aafd5aaead5789
# git cherry-pick -s -S -x 096a7afd37d688332b961994116a101d9f3ffab9 # SKIPPED, requires swarmkit backport to bump_v19.03 branch
git cherry-pick -s -S -x 52e0dfef9090fa3c6003115a2c82238b189ebe42

# https://github.com/docker/swarmkit/pull/2808 Fix flaky tests
git cherry-pick -s -S -x b79adac339173bf8bb6de6d0a061a97973c4b62b

@thaJeztah thaJeztah added this to the 19.03.3 milestone Sep 1, 2019
@thaJeztah
Copy link
Member Author

@tonistiigi @dperny PTAL

@dperny perhaps you can have a look at backporting the changes in swarmkit for 19.03, so that we can update the vendor in a follow up? I updated the PR description of moby#39531 (comment) to show what changes were in that bump; probably only the last two of that are really related to this, but the gometalinter and flaky test changes look like they won't hurt to have in the "bump" branches

full diff: https://github.com/docker/swarmkit/compare/fb584e7b501ec4683b5c3e62476d76b8a7e7d9f6...7dded76ec532741c1ad9736cd2bb6d6661f0a386

includes:

- docker/swarmkit#2808 Fix flaky tests
- docker/swarmkit#2866 Swap gometalinter for golangci-lint
- docker/swarmkit#2869 Increase max recv gRPC message size to initialize connection broker
- docker/swarmkit#2870 Fix update out of sequence

@thaJeztah
Copy link
Member Author

opened moby/swarmkit#2889 to backport those fixes in swarmkit

@thaJeztah
Copy link
Member Author

Janky failed on a flaky https://ci.docker.com/public/blue/rest/organizations/jenkins/pipelines/engine/branches/PR-345/runs/3/nodes/149/log/?start=0

tracked through moby#39426
and on master is currently skipped (see moby#39453)

[2019-09-06T21:43:12.379Z] --- FAIL: TestServiceRemoveKeepsIngressNetwork (31.44s)
[2019-09-06T21:43:12.379Z]     service_test.go:232: Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/1/TestServiceRemoveKeepsIngressNetwork
[2019-09-06T21:43:12.379Z]     daemon.go:321: [df2ee9961c8e2] waiting for daemon to start
[2019-09-06T21:43:12.379Z]     daemon.go:339: [df2ee9961c8e2] daemon started
[2019-09-06T21:43:12.379Z]     service_test.go:237: timeout hit after 30s: ingress not ready: 0 peers and 0 containers
[2019-09-06T21:43:12.379Z]     daemon.go:449: [df2ee9961c8e2] Stopping daemon
[2019-09-06T21:43:12.379Z]     daemon.go:288: [df2ee9961c8e2] exiting daemon
[2019-09-06T21:43:12.379Z]     daemon.go:436: [df2ee9961c8e2] Daemon stopped
[2019-09-06T21:43:12.379Z] === RUN   TestServiceWithDataPathPortInit
[2019-09-06T21:43:12.635Z] PASS: docker_cli_inspect_test.go:309: DockerSuite.TestInspectSizeFlagContainer	0.908s
[2019-09-06T21:43:13.995Z] PASS: docker_cli_inspect_test.go:56: DockerSuite.TestInspectStatus	1.095s
[2019-09-06T21:43:15.878Z] PASS: docker_cli_inspect_test.go:353: DockerSuite.TestInspectStopWhenNotFound	1.529s
[2019-09-06T21:43:16.801Z] PASS: docker_cli_inspect_test.go:320: DockerSuite.TestInspectTemplateError	0.798s
[2019-09-06T21:43:18.163Z] PASS: docker_cli_inspect_test.go:263: DockerSuite.TestInspectTimesAsRFC3339Nano	1.092s
[2019-09-06T21:43:18.418Z] --- PASS: TestServiceWithDataPathPortInit (6.01s)
[2019-09-06T21:43:18.418Z]     service_test.go:337: Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/1/TestServiceWithDataPathPortInit
[2019-09-06T21:43:18.418Z]     daemon.go:321: [d0c42b6ea3887] waiting for daemon to start
[2019-09-06T21:43:18.418Z]     daemon.go:339: [d0c42b6ea3887] daemon started
[2019-09-06T21:43:18.418Z]     daemon.go:449: [d0c42b6ea3887] Stopping daemon
[2019-09-06T21:43:18.418Z]     daemon.go:288: [d0c42b6ea3887] exiting daemon
[2019-09-06T21:43:18.418Z]     daemon.go:436: [d0c42b6ea3887] Daemon stopped
[2019-09-06T21:43:18.418Z]     service_test.go:365: Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/1/TestServiceWithDataPathPortInit
[2019-09-06T21:43:18.418Z]     daemon.go:321: [d4ec035907cb1] waiting for daemon to start
[2019-09-06T21:43:18.418Z]     daemon.go:339: [d4ec035907cb1] daemon started
[2019-09-06T21:43:18.418Z]     daemon.go:449: [d4ec035907cb1] Stopping daemon
[2019-09-06T21:43:18.418Z]     daemon.go:288: [d4ec035907cb1] exiting daemon
[2019-09-06T21:43:18.418Z]     daemon.go:436: [d4ec035907cb1] Daemon stopped

@thaJeztah
Copy link
Member Author

opened #350 for that remaining flaky test

thaJeztah added a commit to thaJeztah/docker that referenced this pull request Sep 12, 2019
…3 branch)

full diff: moby/swarmkit@4fb9e96...bbe3418

changes included:

- moby/swarmkit#2889 [19.03 backport] Fix update out of sequence and increase max recv gRPC message size for nodes and secrets

Which relates to

- moby#39531 integration-cli: fix swarm tests flakiness
- docker-archive#345 [19.03 backport] integration-cli: fix swarm tests flakiness

And includes backports of

- moby/swarmkit#2808 Fix flaky tests
- moby/swarmkit#2866 Swap gometalinter for golangci-lint
- moby/swarmkit#2869 Increase max recv gRPC message size to initialize connection broker
 - related / similar to moby#38103 / docker-archive#102 cluster: set bigger grpc limit for array requests
 - related / similar to moby#39306 Increase max recv gRPC message size for nodes and secrets
 - fixes moby/swarmkit#2733 Error generated when messages size is too big
- moby/swarmkit#2870 Fix update out of sequence

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Member Author

opened #353 for the swarmkit bump, and rebased this PR on top of that

@thaJeztah
Copy link
Member Author

this is all green, but I'll rebase to get rid of the commit from #353

@thaJeztah
Copy link
Member Author

rebased

docker-jenkins pushed a commit to docker-archive/docker-ce that referenced this pull request Sep 12, 2019
…3 branch)

full diff: moby/swarmkit@4fb9e96...bbe3418

changes included:

- moby/swarmkit#2889 [19.03 backport] Fix update out of sequence and increase max recv gRPC message size for nodes and secrets

Which relates to

- moby/moby#39531 integration-cli: fix swarm tests flakiness
- docker-archive/engine#345 [19.03 backport] integration-cli: fix swarm tests flakiness

And includes backports of

- moby/swarmkit#2808 Fix flaky tests
- moby/swarmkit#2866 Swap gometalinter for golangci-lint
- moby/swarmkit#2869 Increase max recv gRPC message size to initialize connection broker
 - related / similar to moby/moby#38103 / docker-archive/engine#102 cluster: set bigger grpc limit for array requests
 - related / similar to moby/moby#39306 Increase max recv gRPC message size for nodes and secrets
 - fixes moby/swarmkit#2733 Error generated when messages size is too big
- moby/swarmkit#2870 Fix update out of sequence

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: f7dbee3eeaa1dd218116f85b8f60361acbd5b214
Component: engine
tonistiigi and others added 3 commits September 16, 2019 13:45
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 3df1095)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 52e0dfe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
TestSwarmClusterRotateUnlockKey had been identified as a flaky test. It
turns out that the test code was wrong: where we should have been
checking the string output of a command, we were instead checking the
value of the error. This means that the error case we were expecting was
not being matched, and the test was failing when it should have just
retried.

Signed-off-by: Drew Erny <drew.erny@docker.com>
(cherry picked from commit b79adac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Member Author

all green now; I'm bringing this one in

@thaJeztah thaJeztah merged commit b262d40 into docker-archive:19.03 Sep 16, 2019
@thaJeztah thaJeztah deleted the 19.03_backport_swarm_flaky branch September 16, 2019 14:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
4 participants