Skip to content
This repository has been archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] assorted test-fixes #401

Merged
merged 78 commits into from Oct 28, 2019

Conversation

thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Oct 8, 2019

backports of the following PRs (more details about some conflicts when cherry-picking below):

Steps followed:

# https://github.com/moby/moby/pull/38266 Configure log-format earlier, and small refactor
git cherry-pick -s -S -x 1edf943dc7b6fc56050ac028a79ba4dc83e7fbe7

# https://github.com/moby/moby/pull/37516 migrate container wait integration tests from integration-cli to integration/container package
git cherry-pick -s -S -x 7073aa8a3a0b8e7fdbda5ad7c174fb66cfd384fc

Only including the new test-utilities added in moby/moby/38003, as they are used by other tests now
git cherry-pick -s -S -x 596cdffb9fdea5323ccd7196f196544912b59c1f


git reset -- \
	api/server/router/container/container_routes.go \
	api/swagger.yaml \
	api/types/mount/mount.go \
	container/mounts_unix.go \
	daemon/cluster/convert/container.go \
	daemon/oci_linux.go \
	daemon/volumes_unix.go \
	volume/mounts/linux_parser.go \
	docs/api/version-history.md \
	integration/container/mounts_linux_test.go


git checkout -- \
	api/server/router/container/container_routes.go \
	api/swagger.yaml \
	api/types/mount/mount.go \
	container/mounts_unix.go \
	daemon/cluster/convert/container.go \
	daemon/oci_linux.go \
	daemon/volumes_unix.go \
	volume/mounts/linux_parser.go \
	docs/api/version-history.md \
	integration/container/mounts_linux_test.go

Including these PRs to help getting a cleaner cherry-pick, also, these update the Dockerfile
to use a more current version or CRIU, which may be needed when we update to newer Go versions

The second PR disables the test that was added, so this is mostly dead code (for now)

# https://github.com/moby/moby/pull/38452 integration/container: add a base test for C/R
git cherry-pick -s -S -x \
	bd4fff38cc7eef5b9ea4051217f6221790f48018 \
	0b96bf891c64d6c707a0831466c53c16c8d99e02

Minor conflict in the first commit, because we already backported https://github.com/moby/moby/pull/39635

# https://github.com/moby/moby/pull/38984 Temporarily disable TestCheckpoint
git cherry-pick -s -S -x 23fec5025dacbfef7a1c497147c7c0ff0dcb729f


# https://github.com/moby/moby/pull/39332 Integration: change signatures to fix golint warnings
git cherry-pick -s -S -x \
	caec45a37faf351afc82cb6228ea2e2b48589432 \
	b4c46b0dac0d74fed8b8adc56cbb7f55788e4916 \
	9f9b4290b92bab7d0adca1f3a2bf404f013d7ef5 \
	123e29f44aa9fb39f7ee5cb42c94b4d7bbe8ba82 \
	dac5710b689fc6e0614c6b20a11017ad30e907f8

# https://github.com/moby/moby/pull/39359 integration: get tests to compile again
git cherry-pick -s -S -x a2812895154da7a32a1b24a0535b5cba4088eb89

# revert 584c0857ab21895e62feac686448085113c6c977 (temporarily added in https://github.com/docker/engine/pull/286)
git revert -s -S 584c0857ab21895e62feac686448085113c6c977

# revert 614daf117112e8c9576967764281cc6fe617bbb2 (temporarily added in https://github.com/docker/engine/pull/320)
git revert -s -S 614daf117112e8c9576967764281cc6fe617bbb2


first commit (caec45a37faf351afc82cb6228ea2e2b48589432):
	deleted by us:   integration/container/run_linux_test.go -> because https://github.com/moby/moby/pull/37043 is not yet in 18.09
	both modified:   integration/network/service_test.go     -> because https://github.com/moby/moby/pull/38102 is not yet in 18.09

second commit (b4c46b0dac0d74fed8b8adc56cbb7f55788e4916):

	both modified:   integration/container/ipcmode_linux_test.go      -> because TestIpcModeOlderClient is not yet in 18.09
	deleted by us:   integration/container/run_cgroupns_linux_test.go -> because https://github.com/moby/moby/pull/38377 is not in 18.09
	both modified:   integration/internal/container/container.go      -> because https://github.com/moby/moby/pull/38377 is not in 18.09

third commit (9f9b4290b92bab7d0adca1f3a2bf404f013d7ef5):

	both modified:   integration/container/mounts_linux_test.go       -> because https://github.com/moby/moby/pull/38003 is not yet in 18.09
	deleted by us:   integration/container/run_cgroupns_linux_test.go -> because https://github.com/moby/moby/pull/38377 is not in 18.09
	deleted by us:   integration/container/run_linux_test.go          -> because https://github.com/moby/moby/pull/37043 is not yet in 18.09
	both modified:   integration/container/update_linux_test.go       -> because https://github.com/moby/moby/pull/32519 is not yet in 18.09


# https://github.com/moby/moby/pull/37836 Fix flaky TestServiceWithDefaultAddressPoolInit
git cherry-pick -s -S -x 88578aa9e9c3deaf634acc53d1460ac3e4d17e03

# https://github.com/moby/moby/pull/39966 Add TC to check dynamic subnet for ingress network
# NOTE: not yet merged upstream, so git commit may not be found there
git cherry-pick -s -S -x 084f5ab5c8ccc7d5513d1b751bac505e5b5e8221

# https://github.com/moby/moby/pull/38411 integration-cli/build: don't panic ;)
git cherry-pick -s -S -x 7e7ff2a033abf87438efe472b5b4c823be09ad48

# https://github.com/moby/moby/pull/38853 integration-cli: don't build -test images if they already exist
git cherry-pick -s -S -x \
	d283c7fa2b93e00d4e1b0feaee99028b00dd775d \
	175b1d783013b7e73f117f512e1179ee3b66a697 \
	ba0afa6ba865c264e0fbcb5d5e4590d8f2748f72


# https://github.com/moby/moby/pull/39430 Integration: remove redundant checks in IPVLAN tests
git cherry-pick -s -S -x \
	1e4bd2623a37b8083a7c5f8a93068be2c63f491c \
# skipped because it's still experimental in 18.09
#	dae9bac6757af3027b06a983b8448ffa962cc071 \
	93b28677bffcdaf299969c4ff1603b7d07b45fbc \
	4060a7026c0ffa2983ab1fa41d43da4937ef63c6

# https://github.com/moby/moby/pull/39431 Integration: remove redundant kernel version check for MACVlan
git cherry-pick -s -S -x \
	691eb142561818b0ca2255e2e340cbea6b91d80c \
	316e16618f8c794b8e56e223ed0bed15e1f4be24


# https://github.com/moby/moby/pull/39468 integration-cli: remove ExecSupport check
git cherry-pick -s -S -x 7204341950c6c8f0a66f9bb0b082217dc0ce6ddb

# https://github.com/moby/moby/pull/39469 integration-cli: remove defaultSleepImage constant
git cherry-pick -s -S -x 27f432ca57fb6d4d0409fc3c5358b74feae228cc

# https://github.com/moby/moby/pull/39470 integration-cli: remove unused requirements utils
git cherry-pick -s -S -x c887b09abc7d4ae149a9c314f74f9c351ac4cba2

# https://github.com/moby/moby/pull/39500 Add (hidden) flags to set containerd namespaces

http://github.com/containerd/containerd/compare/9754871865f7fe2f4e74d43e2fc7ccd237edcbce...3f42445e38d1081f4b8c3b8d7d1ed1860198ed7a

git cherry-pick -s -S -x 5accd8263497809ec8bab1d01251986d139bb380 
-> taken from https://github.com/moby/moby/pull/37674 Revendor Microsoft/hcsshim and go-winio, plus container/containerd
these changes were missed, because https://github.com/docker/engine/pull/60 did a manual vendor update, therefore not including the local changes

git cherry-pick -s -S -x 24ad2f486d92681080a8e257760b047f8de2c71c


	both modified:   cmd/dockerd/config.go -> minor conflict, because https://github.com/moby/moby/pull/38050 changed the signature to allow returning an error. Resolved by suppressing the error from "markhidden"


# https://github.com/moby/moby/pull/39509 integration: fix cleanup of raft data
git cherry-pick -s -S -x 6a64a4deecf5b7519d3748023765f79628059f80


# https://github.com/moby/moby/pull/39105 Optimize test daemon startup
git cherry-pick -s -S -x \
	20ea8942b86b271f49c25aa70c53f7ca38571a2c \
	595987fd082165b0c5739993b374bb5b6fa3f466

second commit added logs to code that isn't yet in 18.09, so I skipped those parts:

	both modified:   cmd/dockerd/daemon.go
	both modified:   cmd/dockerd/daemon_unix.go



# https://github.com/moby/moby/pull/39452 Improve select for daemon restart tests
git cherry-pick -s -S -x 402433a5e4b8a74cab404658f34e6520e71df00a

# https://github.com/moby/moby/pull/39510 Don't log test initial test daemon ping failures
git cherry-pick -s -S -x 15675e28f1e3ba25f3abd099c14a7800c8bae2b8

# https://github.com/moby/moby/pull/39517 Fix Microsecond -> Millisecond
git cherry-pick -s -S -x 5d818213ff3b9f8cda8e4fb3b071bef8fab782ad

# https://github.com/moby/moby/pull/39516 DockerSwarmSuite lock portIndex to work around race
git cherry-pick -s -S -x c096225e8ee6cb36e363ac38de4e4a566cf09918

# https://github.com/moby/moby/pull/39552 Be more conservative for Windows in TestFrequency for Splunk
git cherry-pick -s -S -x a5c420ac54f07dbc84bcdea91b83364a25f1c921

# https://github.com/moby/moby/pull/38428 Testing: create new daemon (only) if needed
git cherry-pick -s -S -x b3407d20295890c4e66aaabc043d9a4a872b6130

# https://github.com/moby/moby/pull/39554 integration: run build session tests on non-experimental
git cherry-pick -s -S -x becd29c6651dffc253027e15f903ae7e7c918594

# https://github.com/moby/moby/pull/39557 integration-cli: increase healthcheck timeout
    # relates to https://github.com/moby/moby/issues/39499#issuecomment-512621697 flaky DockerSwarmSuite tests

git cherry-pick -s -S -x 8c9362857f352548b94bc39c675d558b4da0d3b3

# https://github.com/moby/moby/pull/39560 Retry service updates on out of sequence errors
git cherry-pick -s -S -x 1de914695b7d0c9affc97a6da4198548da2f5f78

# https://github.com/moby/moby/pull/39569 integration-cli: remove redundant "testrequires"
git cherry-pick -s -S -x 7f37d99ef50b4046284ecb3c3b290319acf11405

# https://github.com/moby/moby/pull/39602 Add `FromClient` to test env execution
git cherry-pick -s -S -x 1381956499d357dcae47dd1239d4f35b176fea7d

# https://github.com/moby/moby/pull/39603 Better logging for swarm tests
git cherry-pick -s -S -x b0fe0dff7a5b7f70e2d08e59f4773c40e48e5c0d

# https://github.com/moby/moby/pull/39604 hack: Remove inContainer check, it wasn't useful
git cherry-pick -s -S -x f5cd8fdd446750a749182f3410adc6f1968053ca

# https://github.com/moby/moby/pull/39671 Fix flaky TestServiceWithDefaultAddressPoolInit
    # fixes https://github.com/moby/moby/issues/38514 Flaky test: TestServiceWithDefaultAddressPoolInit
 
git cherry-pick -s -S -x \
	f3a3ea0d3c7f4d4da035db871d3c8a8bbb51371f \
	a65dee30fc36ed974940b699ed22ab9242eac6cf


	both modified:   integration/network/service_test.go -> because https://github.com/moby/moby/pull/38102 is not in 18.09

# https://github.com/moby/moby/pull/38783 Windows: Disable 2 restart tests on Hyper-V isolation
git cherry-pick -s -S -x faaffd5d6d7f143619c2bbefc818c40ffbf92523

# https://github.com/moby/moby/pull/39688 integration-cli: Skip windows specific isolation requirements on non-windows
git cherry-pick -s -S -x b469933b063169718987865b8b1215cb7befd1a6

minor conflicts, because 18.09 didn't move from go-check to gotest

	both modified:   integration-cli/docker_cli_restart_test.go
	both modified:   integration-cli/docker_cli_run_test.go

# https://github.com/moby/moby/pull/39743 Dockerfile: update CRIU to v3.12
git cherry-pick -s -S -x 00ad0222cef8024f19394e2e9d5dc0e05f0ec982

# https://github.com/moby/moby/pull/39800 pkg/term: refactor TestEscapeProxyRead
git cherry-pick -s -S -x 556d26c07d068d92fd896428ab4ac890554239d7

# https://github.com/moby/moby/pull/39804 integration: windows.ps1: turn defender error into a warning
git cherry-pick -s -S -x 31885181fcc0ca0cacd6c12f6c64ac553ff2bc2b

# https://github.com/moby/moby/pull/36537 LCOW: Log stderr on failures to ease diagnosis
to assist in cherry-picking https://github.com/moby/moby/pull/39817
git cherry-pick -s -S -x 63f9c7784b7c6a726c8c668d68f9c8cb13e19ffb

# https://github.com/moby/moby/pull/39817 TestDispatch: refactor to use subtests again, and fix linting (structcheck)
git cherry-pick -s -S -x a3f9cb5b635a76484a926289bfa4fc6feee1763a

# https://github.com/moby/moby/pull/39819 integration-cli: getContainerCount() fix trimming prefix
git cherry-pick -s -S -x 02c9b0674fe2cf43def3a1e625cab74a0db8136a

# https://github.com/moby/moby/pull/39831 Fix docker inspect for dutimgVersion
git cherry-pick -s -S -x 52a53e2587e34d45b1d4a9b7b7d307ad51c5b0b0

# https://github.com/moby/moby/pull/39844 Fix some bashisms/non-standard comparisons in integration tests
git cherry-pick -s -S -x \
	dbde4786e48531f095f9c3ecaff0f57b838abefc \
	32f1c651623421ee1ac480b200d34025a74436bb

# https://github.com/moby/moby/pull/39836 hack/make/binary-daemon: fix some linting issues
git cherry-pick -s -S -x 70d36778252124d7cec0489ed98d1bac475d9727
some conflicts, because rootless kit is not in 18.09

# https://github.com/moby/moby/pull/39853 TestRunInteractiveWithRestartPolicy: use `icmd.RunCmd` instead `icmd.StartCmd`
git cherry-pick -s -S -x e6fce00ec83df2f23523b836f647b8f3df97953f

# https://github.com/moby/moby/pull/39854 hack/ci/windows.ps1: explicitly set exit code to result of tests
git cherry-pick -s -S -x \
	8e8c52c4abe011a4cf3334da0726ef1fc0d17b14 \
	7eb522a2350d759cf6a9aad493ac1b8ffc3d3335

# https://github.com/moby/moby/pull/39876 integration-cli: update TestCreateWithWorkdir for Hyper-V isolation
git cherry-pick -s -S -x ac9ef840ef94ff66266d3d8b9d32caf570d3b93f

# https://github.com/moby/moby/pull/39877 hack/ci/windows.ps1: add support for DOCKER_STORAGE_OPTS
git cherry-pick -s -S -x b6f596c4112818109441c84d313cf38fa06d6768

# https://github.com/moby/moby/pull/39878 hack/ci/windows.ps1: stop tailing logs after stopping the daemon
git cherry-pick -s -S -x e1636ad5fa1351c3edc20dd9006fb4d6f63c9f69

# https://github.com/moby/moby/pull/39884 Fix Service TTY test so signal handlers work
git cherry-pick -s -S -x e6c5563ae9d083cc72fcf84796c542047ac0a299

# https://github.com/moby/moby/pull/39886 integration-cli: don't load busybox image on every daemon start
git cherry-pick -s -S -x \
	8fc23588f1aa14054f1134e60dd996220c595363 \
	ead3f4e7c8b7add7beb2de44649b38f41947180f

# https://github.com/moby/moby/pull/39887 Windows: skip flaky TestLogBlocking 
git cherry-pick -s -S -x 6c75c862403c98138ab0f7811f6ba9113f9e5d61

# https://github.com/moby/moby/pull/39968 awslogs: fix flaky TestLogBlocking unit test
git cherry-pick -s -S -x fd94bae0b8acc451a79667dd1cef4c583d532187

# https://github.com/moby/moby/pull/39889 Fix more signal handling issues in tests
git cherry-pick -s -S -x fcd65ebf49a858c4f6223d1b1db728f7400a3b6d

	both modified:   integration-cli/docker_cli_userns_test.go -> because https://github.com/moby/moby/pull/39902 is not in 18.09

# https://github.com/moby/moby/pull/39945 Disable TestPsListContainersFilterExited (Windows)
TestPsListContainersFilterExited
git cherry-pick -s -S -x 7de4e130898fc1cf74f8c4fec24416c3f7d3589b

# https://github.com/moby/moby/pull/39965 integration-cli: run goimports
git cherry-pick -s -S -x 5b7347c312161da0b64c551fee70c111ff5c9ec6

# pushed an extra commit here, to change `skip.If` to `testRequires`

# https://github.com/moby/moby/pull/39930 hack/test/unit: fix custom TESTFLAGS not working
git cherry-pick -s -S -x 0620990307aaf8ada706ffb6c5dc0628c92d84af

@thaJeztah thaJeztah added this to the 18.09.11 milestone Oct 8, 2019
@thaJeztah
Copy link
Member Author

00:09:20.525  integration/service/network_test.go:89:20:warning: cannot use t (variable of type *testing.T) as context.Context value in argument to net.CreateNoError: missing method 
00:12:16.313  # github.com/docker/docker/integration/container [github.com/docker/docker/integration/container.test]
00:12:16.313  integration/container/checkpoint_test.go:53:39: undefined: "github.com/docker/docker/integration/internal/container".WithMount
00:12:16.313  integration/container/copy_test.go:108:25: cannot use t (type *testing.T) as type context.Context in argument to "github.com/docker/docker/integration/internal/container".Create:
00:12:16.313  	*testing.T does not implement context.Context (missing Deadline method)
00:12:16.313  integration/container/copy_test.go:108:25: cannot use ctx (type context.Context) as type *testing.T in argument to "github.com/docker/docker/integration/internal/container".Create
00:12:16.313  integration/container/health_test.go:45:21: cannot use t (type *testing.T) as type context.Context in argument to "github.com/docker/docker/integration/internal/container".Run:
00:12:16.313  	*testing.T does not implement context.Context (missing Deadline method)
00:12:16.313  integration/container/health_test.go:45:21: cannot use ctx (type context.Context) as type *testing.T in argument to "github.com/docker/docker/integration/internal/container".Run

@thaJeztah thaJeztah changed the title [18.09 backport] assorted test-fixes [WIP][18.09 backport] assorted test-fixes Oct 8, 2019
@thaJeztah thaJeztah force-pushed the 18.09_backport_testfixes branch 3 times, most recently from 70d1bd5 to fee45b5 Compare October 8, 2019 15:14
@thaJeztah
Copy link
Member Author

Failures:

https://ci.docker.com/public/job/engine/job/PR-401/5/execution/node/164/log/?consoleFull

00:12:37.125  FAIL: docker_cli_daemon_test.go:2126: DockerDaemonSuite.TestDaemonStartWithoutColors


00:16:54.975  FAIL: docker_api_swarm_test.go:292: DockerSwarmSuite.TestAPISwarmLeaderElection

thaJeztah and others added 15 commits October 23, 2019 18:10
Some messages are logged before the logrus format was set,
therefore resulting in inconsistent log-message formatting
during startup;

Before this patch;

```
dockerd --experimental
WARN[0000] Running experimental build
INFO[2018-11-24T11:24:05.615249610Z] libcontainerd: started new containerd process  pid=132
INFO[2018-11-24T11:24:05.615348322Z] parsed scheme: "unix"                         module=grpc
...
```

With this patch applied;

```
dockerd --experimental
WARN[2018-11-24T13:41:51.199057259Z] Running experimental build
INFO[2018-11-24T13:41:51.200412645Z] libcontainerd: started new containerd process  pid=293
INFO[2018-11-24T13:41:51.200523051Z] parsed scheme: "unix"                         module=grpc
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1edf943)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
…egration/container package

Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
(cherry picked from commit 7073aa8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Test changes, taken from 596cdff

git cherry-pick -s -S -x 596cdff

git reset -- \
	api/server/router/container/container_routes.go \
	api/swagger.yaml \
	api/types/mount/mount.go \
	container/mounts_unix.go \
	daemon/cluster/convert/container.go \
	daemon/oci_linux.go \
	daemon/volumes_unix.go \
	volume/mounts/linux_parser.go \
	docs/api/version-history.md \
	integration/container/mounts_linux_test.go

git checkout -- \
	api/server/router/container/container_routes.go \
	api/swagger.yaml \
	api/types/mount/mount.go \
	container/mounts_unix.go \
	daemon/cluster/convert/container.go \
	daemon/oci_linux.go \
	daemon/volumes_unix.go \
	volume/mounts/linux_parser.go \
	docs/api/version-history.md \
	integration/container/mounts_linux_test.go

mount: add BindOptions.NonRecursive (API v1.40)

This allows non-recursive bind-mount, i.e. mount(2) with "bind" rather than "rbind".

Swarm-mode will be supported in a separate PR because of mutual vendoring.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit 596cdff)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrei Vagin <avagin@gmail.com>
(cherry picked from commit bd4fff3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A container checkpoint directory doesn't have config.json.

Signed-off-by: Andrei Vagin <avagin@gmail.com>
(cherry picked from commit 0b96bf8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 23fec50)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
    Line 30: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit caec45a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
Line 25: warning: context.Context should be the first parameter of a function (golint)
Line 44: warning: context.Context should be the first parameter of a function (golint)
Line 52: warning: context.Context should be the first parameter of a function (golint)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b4c46b0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
    Line 59: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9f9b429)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
    Line 30: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 123e29f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
    Line 441: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dac5710)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit a281289)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 584c085.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 614daf1.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like TestServiceWithDefaultAddressPoolInit is failing
randomly in CI. I am not able to reproduce the issue locally
but this has been reported few times.  So I tried to modify
code and see if I can fix the random failure.

Signed-off-by: selansen <elango.siva@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 88578aa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah thaJeztah changed the title [WIP][18.09 backport] assorted test-fixes [18.09 backport] assorted test-fixes Oct 23, 2019
@thaJeztah
Copy link
Member Author

Github links to an old job; this is the current one; https://ci.docker.com/public/blue/organizations/jenkins/engine/detail/PR-401/15/pipeline

@thaJeztah
Copy link
Member Author

OK, so it's failing on: https://ci.docker.com/public/blue/rest/organizations/jenkins/pipelines/engine/branches/PR-401/runs/15/nodes/215/log/?start=0


[2019-10-23T16:46:56.049Z] === FAIL: s390x.integration.network TestServiceWithDefaultAddressPoolInit (7.27s)
[2019-10-23T16:46:56.049Z]     service_test.go:321: Creating a new daemon
[2019-10-23T16:46:56.049Z]     service_test.go:321: Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/TestServiceWithDefaultAddressPoolInit
[2019-10-23T16:46:56.049Z]     daemon.go:324: [dcde7476c28e8] waiting for daemon to start
[2019-10-23T16:46:56.049Z]     daemon.go:324: [dcde7476c28e8] waiting for daemon to start
[2019-10-23T16:46:56.049Z]     daemon.go:352: [dcde7476c28e8] daemon started
[2019-10-23T16:46:56.049Z]     service_test.go:351: TestServiceWithDefaultAddressPoolInit: NetworkInspect: {Name:sthiraTestServiceWithDefaultAddressPoolInit ID:nkcjqvbh8nomm6nf5tytlwtv1 Created:2019-10-23 16:46:49.567513403 +0000 UTC Scope:swarm Driver:overlay EnableIPv6:false IPAM:{Driver:default Options:map[] Config:[{Subnet:20.20.1.0/24 IPRange: Gateway:20.20.1.1 AuxAddress:map[]}]} Internal:false Attachable:false Ingress:false ConfigFrom:{Network:} ConfigOnly:false Containers:map[edf03ea630571e41a4f5a054770650833ebce0a2b44dc66e1e8ea9500bdbb70e:{Name:TestServiceTestServiceWithDefaultAddressPoolInit.1.d7srxtf0si7ykkagzawsieq2n EndpointID:75538917742e31c32d7d0d2a36806d078a3be93d4b796fd54e7ffa916e54a431 MacAddress:02:42:14:14:01:03 IPv4Address:20.20.1.3/24 IPv6Address:} lb-sthiraTestServiceWithDefaultAddressPoolInit:{Name:sthiraTestServiceWithDefaultAddressPoolInit-endpoint EndpointID:557c2ffa568fd37c64ccfe278aecbfe61d8910102ea7b7ce086cdccc23aa568b MacAddress:02:42:14:14:01:04 IPv4Address:20.20.1.4/24 IPv6Address:}] Options:map[com.docker.network.driver.overlay.vxlanid_list:4097] Labels:map[] Peers:[{Name:b14bccd80365 IP:127.0.0.1}] Services:map[TestServiceTestServiceWithDefaultAddressPoolInit:{VIP:20.20.1.2 Ports:[] LocalLBIndex:256 Tasks:[{Name:TestServiceTestServiceWithDefaultAddressPoolInit.1.d7srxtf0si7ykkagzawsieq2n EndpointID:75538917742e31c32d7d0d2a36806d078a3be93d4b796fd54e7ffa916e54a431 EndpointIP:20.20.1.3 Info:map[Host IP:127.0.0.1]}]}]}
[2019-10-23T16:46:56.049Z]     service_test.go:353: assertion failed: 20.20.1.0/24 (out.IPAM.Config[0].Subnet string) != 20.20.0.0/24 (string)
[2019-10-23T16:46:56.049Z]     daemon.go:460: [dcde7476c28e8] Stopping daemon
[2019-10-23T16:46:56.049Z]     daemon.go:295: [dcde7476c28e8] exiting daemon
[2019-10-23T16:46:56.049Z]     daemon.go:447: [dcde7476c28e8] Daemon stopped

And I just notice that this is an actual failure because swarmkit was updated with a fix to swarmkit that requires this test-change; moby#39966

Fix in swarmkit is moby/swarmkit#2890, and that got merged as part of #346

We need to apply this patch:

+	assert.Equal(t, out.IPAM.Config[0].Subnet, "20.20.1.0/24")
+
+ 	// Also inspect ingress network and make sure its in the same subnet
+ 	out, err = cli.NetworkInspect(ctx, "ingress", types.NetworkInspectOptions{Verbose: true})
+ 	assert.NilError(t, err)
+ 	assert.Assert(t, len(out.IPAM.Config) > 0)

Arko Dasgupta and others added 3 commits October 23, 2019 20:10
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit 084f5ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A lack of check in the test code can lead to a panic due to
`len(ids)` being `0`.

Avoid the panic by adding appropriate checks. Note `Assert()` should be
used rather than `Check()` as if it fails we should not proceed with the
test.

Originally found in moby#38404.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 7e7ff2a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It has been declared deprecated by the author, and has a knack for
false-positives (as well as giving bad advice when it comes to APIs --
which is quite clear when looking at "nolint: interfacer" comments).

Signed-off-by: Aleksa Sarai <asarai@suse.de>
(cherry picked from commit d283c7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah and others added 19 commits October 23, 2019 20:12
`TestBuildBuildTimeArgEnv` and `TestBuildBuildTimeArgEmptyValVariants` were
using non-standard comparisons.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dbde478)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 32f1c65)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Add quotes to prevent word splitting in `cp` statement (SC2046)
- Replace legacy back tics with `$()`
- Replace `which` with `command -v` (SC2230)
- Fix incorrect (`==`) comparison

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 70d3677)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Use `cli.Docker` instead `dockerCmdWithResult`.

Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
(cherry picked from commit e6fce00)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Trying to see if this helps with the cleanup step exiting in CI, but
Jenkins continuing to wait for the script to end afterwards.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e8c52c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
…ins runs this script

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7eb522a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Hyper-V isolated containers do not allow file-operations on a
running container. This test currently uses `docker cp` to verify
that the WORKDIR was automatically created, which cannot be done
while the container is running.

```
FAIL: docker_cli_create_test.go:302: DockerSuite.TestCreateWithWorkdir

assertion failed:
Command:  d:\CI-7\CI-f3768a669\binary\docker.exe cp foo:c:\home\foo\bar c:\tmp
ExitCode: 1
Error:    exit status 1
Stdout:
Stderr:   Error response from daemon: filesystem operations against a running Hyper-V container are not supported

Failures:
ExitCode was 1 expected 0
Expected no error
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ac9ef84)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b6f596c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There's already a step in  "Nuke Everything", but lets' stop it
after stopping the daemon as well

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1636ad)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Noticed this test container not exiting correctly while debugging
another issue. Before this change, signals were being eaten by bash, now
they are hanlded by top. This cuts the test time in half since it
doesn't have to wait for docker to SIGKILL it.

Old:
PASS: docker_cli_swarm_test.go:840: DockerSwarmSuite.TestSwarmServiceTTY	18.997s

New:
PASS: docker_cli_swarm_test.go:840: DockerSwarmSuite.TestSwarmServiceTTY	6.293s

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e6c5563)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The daemon was already created and started with the busybox
image loaded, so there's no need to load the image again.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8fc2358)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Starting the daemon should not load the busybox image again
in most cases, so add a new `StartNodeWithBusybox` function
to be clear that this one loads the busybox image, and use
`StartNode()` for cases where loading the busybox image is
not needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ead3f4e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This test frequently fails on Windows RS1 (mainly), so skipping it
for now on Windows;

```
ok  	github.com/docker/docker/daemon/logger	0.525s	coverage: 43.0% of statements
time="2019-09-09T20:37:35Z" level=info msg="Trying to get region from EC2 Metadata"
time="2019-09-09T20:37:36Z" level=info msg="Log stream already exists" errorCode=ResourceAlreadyExistsException logGroupName= logStreamName= message= origError="<nil>"
--- FAIL: TestLogBlocking (0.02s)
    cloudwatchlogs_test.go:313: Expected to be able to read from stream.messages but was unable to
time="2019-09-09T20:37:36Z" level=error msg=Error
time="2019-09-09T20:37:36Z" level=error msg="Failed to put log events" errorCode=InvalidSequenceTokenException logGroupName=groupName logStreamName=streamName message="use token token" origError="<nil>"
time="2019-09-09T20:37:36Z" level=error msg="Failed to put log events" errorCode=DataAlreadyAcceptedException logGroupName=groupName logStreamName=streamName message="use token token" origError="<nil>"
time="2019-09-09T20:37:36Z" level=info msg="Data already accepted, ignoring error" errorCode=DataAlreadyAcceptedException logGroupName=groupName logStreamName=streamName message="use token token"
FAIL
coverage: 78.2% of statements
FAIL	github.com/docker/docker/daemon/logger/awslogs	0.630s
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6c75c86)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
TestLogBlocking is intended to test that the Log method blocks by
default.  It does this by mocking out the internals of the
awslogs.logStream and replacing one of its internal channels with one
that is controlled by the test.  The call to Log occurs inside a
goroutine.  Go may or may not schedule the goroutine immediately and the
blocking may or may not be observed outside the goroutine immediately
due to decisions made by the Go runtime.  This change adds a small
timeout for test failure so that the Go runtime has the opportunity to
run the goroutine before the test fails.

Signed-off-by: Samuel Karp <skarp@amazon.com>
(cherry picked from commit fd94bae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Found these by doing a `grep -R 'using the force'` on a full test run.
There's still a few more which are running against the main test daemon,
so it is difficult to find which test they belong to.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit fcd65eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On account of being flaky on both RS1 and RS5.

Co-Authored-By: Sebastiaan van Stijn <thaJeztah@users.noreply.github.com>
Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
(cherry picked from commit 7de4e13)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 5b7347c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The 18.09 branch is still using go-check, so patching this to
make it work with go-check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `-test.timeout=5m` was glued directly after the current `TESTFLAGS`,
causing them to be non-functional;

Before:

    make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
    + mkdir -p bundles
    + gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent-test.timeout=5m github.com/docker/docker/pkg/filenotify
    testing: warning: no tests to run
    ok  	github.com/docker/docker/pkg/filenotify	0.003s	coverage: 0.0% of statements [no tests to run]

    DONE 0 tests in 0.298s

After:

    make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
    + mkdir -p bundles
    + gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent -test.timeout=5m github.com/docker/docker/pkg/filenotify
    ok  	github.com/docker/docker/pkg/filenotify	0.608s	coverage: 44.7% of statements

    DONE 1 tests in 0.922s

This was introduced in 42f0a0d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0620990)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Member Author

Added the test-changes from moby#39966 to this PR

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Member Author

Temporarily disabled the flaky-test check, as it was taking up a lot of time; possibly causing the timeout

@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 23, 2019

Now DockerSwarmSuite.TestSwarmVolumePlugin is hanging;

00:43:34.137  FAIL: check_test.go:347: DockerSwarmSuite.TearDownTest
00:43:34.137  
00:43:34.137  check_test.go:352:
00:43:34.137      d.Stop(c)
00:43:34.137  /go/src/github.com/docker/docker/internal/test/daemon/daemon.go:430:
00:43:34.137      t.Fatalf("Error while stopping the daemon %s : %v", d.id, err)
00:43:34.137  ... Error: Error while stopping the daemon d69f674878fa1 : exit status 130
00:43:34.137  
00:43:34.137  
00:43:34.137  ----------------------------------------------------------------------
00:43:34.137  PANIC: docker_cli_swarm_unix_test.go:16: DockerSwarmSuite.TestSwarmVolumePlugin
00:43:34.137  
00:43:34.137  Creating a new daemon
00:43:34.137  Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/DockerSwarmSuite.TestSwarmVolumePlugin
00:43:34.137  [d69f674878fa1] waiting for daemon to start
00:43:34.137  [d69f674878fa1] waiting for daemon to start
00:43:34.137  [d69f674878fa1] daemon started
00:43:34.137  
00:43:34.137  waited for 103.42691ms (out of 30s)
00:43:34.137  waited for 20.588204598s (out of 30s)
00:43:34.137  [d69f674878fa1] Stopping daemon
00:43:34.137  [d69f674878fa1] daemon stop timeout
00:43:34.137  Attempt #2: daemon is still running with pid 5427
00:43:34.137  Attempt #3: daemon is still running with pid 5427
00:43:34.137  Attempt #4: daemon is still running with pid 5427
00:43:34.137  [d69f674878fa1] exiting daemon
00:43:34.137  [d69f674878fa1] Error when stopping daemon: exit status 130
00:43:34.137  ... Panic: Fixture has panicked (see related PANIC)

Logs, with stackdump of that test:
docker.log

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Copy link

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

Copy link

@andrewhsu andrewhsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu andrewhsu merged commit 2c9fee2 into docker:18.09 Oct 28, 2019
@thaJeztah thaJeztah deleted the 18.09_backport_testfixes branch October 28, 2019 18:52
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet