Skip to content
This repository has been archived by the owner on Oct 13, 2023. It is now read-only.

[19.03 backport] Jenkinsfile and related test-changes #328

Merged
merged 71 commits into from
Aug 20, 2019

Conversation

thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Aug 19, 2019

Backports of:

No conflicts

# https://github.com/moby/moby/pull/39294 hack: remove integration-cli-on-swarm
git cherry-pick -s -S -x e7fbe8e457bb15b0750e327d3b29f127dd18fdbd

# https://github.com/moby/moby/pull/39068 Bump docker-py to 4.0.2, and run tests from upstream repository
git cherry-pick -s -S -x 7bfe48cc00318f9d4cf388237855012aafac56b0 \
	968345bc5c0e3817d5d271124a2da11092e7447e \
	ba8f4c7994668992c27619d63e7056770ffb08ad \
	3c15cea650e356f11550e435efa79bea4be88504 \
	980f2813b435166b0fa3509a84f5e435e355f623 \
	6aafe0fd9ed34c4d91adde62c86e47518274a5e8

# https://github.com/moby/moby/pull/39506 integration: organize bundle directory per test
git cherry-pick -s -S -x 9b5e78888dd2ad45748d642573a1e9b347bd9794

# https://github.com/moby/moby/pull/39507 Cleanup "address" when connecting over a UNIX socket
git cherry-pick -s -S -x 92e6e7dd5f7cb71345e9f39e207cda7867c4a90e

# https://github.com/moby/moby/pull/39529 Remove Codecov
git cherry-pick -s -S -x bd5c5373f1b930e6a4f29a77606fa687c2200aeb

# https://github.com/moby/moby/pull/39606 remove DOCKER_EXECDRIVER from Jenkinsfile
git cherry-pick -s -S -x 9d98458fb72c7466879c01a563413bf65acb04b0

# https://github.com/moby/moby/pull/39594 use overlay2 for janky and experimental checks
git cherry-pick -s -S -x ccfaf1ed92a56ac7ffdde9ff00dbf5f1c6cae101

# https://github.com/moby/moby/pull/39592 Added changes to integrate with the new Jenkins ci
git cherry-pick -s -S -x 0ecd6ab30f875184073df81aaf2ec570664957da

# https://github.com/moby/moby/pull/39540 allow running of single integration test
git cherry-pick -s -S -x c222c5ac6f906b6766e8c431a873187ccb957ead

# https://github.com/moby/moby/pull/39535 Set TIMEOUT according to architecture
git cherry-pick -s -S -x 790da6c22327f5c920258ee83126ef4b842e3cbb

# https://github.com/moby/moby/pull/39628 Improvements to the test runners
git cherry-pick -s -S -x abece9b56221bd682f723d5049f7cc047cd522ba

# https://github.com/moby/moby/pull/39638 added entry for running unit tests with junit report
git cherry-pick -s -S -x 42f0a0db75a921145c7f519f7b550e1392890da2

# https://github.com/moby/moby/pull/39656 Jenkinsfile cleanup and improvements
git cherry-pick -s -S -x a95f16ca28d9c02f8c119147f6e9e26e83f8ea77 \
	722d582c920557315d86c6d1b01b069b8f496581 \
	61fd8b7384d060823b538bbd12973326d589ee03 \
	a28f2a2338eac30cdb2138ed66ad59caae18e15f \
	f648964875879c305f47be525eab7705af844612 \
	79713d8d07eb636178f25aa4ace3f88e0adf631c \
	a0bf935f9c40993f62f20623298dc5117aaba38d \
	337d03a5f0074d0716ff6dc46d0a6482d4e50085 \
	9f0e10fe2408bec16a5bb9faac92d4bdf02e84bd \
	b04c769d65e12882283c3fc288cedcf38fb342d2 \
	38977965489f1341993d41819b262d1f29d63c26 \
	355bcf6d480c0a9ee15fc5522e32205922884168

# https://github.com/moby/moby/pull/39661 Some additional Jenkinsfile improvements
git cherry-pick -s -S -x be0e6e9d34d861cd85ef9be452473d944d040253 \
	f51c139792ddcafefecf2b60158cd7aec1202750 \
	f2e09afff436dfd6aa0a36361a5ce77b21ff1c73 \
	6523ced95002377dc6ec9fd62f461a8becee8ac9 \
	a2ad56dfada0f81dec7a72f40623238602cc59d8 \
	c75d7e0e22c16228090b788fa8596b6c81a743de \
	781e79d1fa2c5de4a12e6f8df96b3a6b16ecf5f8 \
	efacee1cdddefa5e69885619a73ade27f6bb3110 \
	0634816c0cabe3b6381661c6eb1640ecffe4f557 \
	f814e0465218d12f4a6d284d20675e1a709e9a71 \
	47ac8a97deac8b1b5e58283ae30994fdfa2446d0 \
	f411be2072e721c49ba2b0f0836d67b9f9785957 \
	1e8ede514e3c4c1033592f2e6e61a0dbbb14bf39 \
	ad28fec1c98b4f28fef0c9e90194497139b7ca08

# https://github.com/moby/moby/pull/39709 Jenkinsfile: remove "experimental" stage
git cherry-pick -s -S -x f8cde0b32d86fa2df71ec65adc3d45f862b3ea33 \
	a43123cab1368fa1dab972481a2951ddbe1e2c66 \
	e856b46cfb29a9b637013369df0f57e6a06e1add

# https://github.com/moby/moby/pull/39714 docker-py: skip PullImageTest::test_pull_invalid_platform
git cherry-pick -s -S -x 6f5c377ddce449bdf297fd4cab1f71e57788f883

# https://github.com/moby/moby/pull/39636 Dockerfile: add back yamllint
git cherry-pick -s -S -x b1723b37212570128b1b403f878cf0064d857f4a

# https://github.com/moby/moby/pull/39685 Jenkinsfile: avoid errors from find
git cherry-pick -s -S -x b283dff3ff872b89dc4f2e70efe3af398bbf9423

# https://github.com/moby/moby/pull/39682 Reduce time of integration tests
git cherry-pick -s -S -x 251c8dca2832e109f61a589e85d80436b6f53a9c \
	13df617d4cc98fe31f3ded78902119a1c1505005 \
	e554fb23c82bd672436800dac3ea9739da8e3377 \
	c2f9d58375fcccf7b6492acaf1c5ff8a705d0725 \
	bdc1c1a02a791fcdf84618939274ef9957cc7ca8 \
	bf70a5975d73ba6833b4a644b4057e9b4ba5f6d5 \
	3564b03fbc5e050b4be04512fd1079a5dd355219 \
	4e2f39cf1447ffd2abd6d0eb1d828c38b3496aa0 \
	0cfc1ec2bdcdb94dcb60dfdfead86aa4aa737195 \
	a049ea1a9301b8804775c719f369a7bc8b5cecc9 \
	ad29f9e471bcad61b66e813845598c7e61e3385c \
	eb30f0ad8448d27cf9303744596ee0310457f0b9

# https://github.com/moby/moby/pull/39719 Jenkinsfile: collect junit.xml for all architectures
git cherry-pick -s -S -x 7f9328ad2e4f2c8aecac9a69f21616943711dc64 \
	e2f5b78e780f2ad0d69f7abdfdada17a0ff66bb2

# https://github.com/moby/moby/pull/39716 docker-py: fix linting, generate junit.xml, save bundles, and various improvements
git cherry-pick -s -S -x 0b3d201892682d4886b5c03d81c1e3d60702bd77 \
	b04cbf10722ec3de5b6ab4b822c6b271e0fd10c8 \
	535e29da057e887000dd4e4d765dc945a5257660 \
	5969bbee79879d44e6ca2b6b5036e5706262bd21 \
	4ddb40ee8abd1b0abbc3887f111605865b0baf5e \
	8b6da9d82f9609c7c3329caa67778edf61933600

# https://github.com/moby/moby/pull/39655 hack: fix mixed tabs/spaces for indentation
git cherry-pick -s -S -x 2cffe9be3d423f2256b4edba50921123363d4b99

# https://github.com/moby/moby/pull/39669 Set locale to fix yamlllint
git cherry-pick -s -S -x b5e5cac0f588b8f8379a9e2574cfe9b09eedf5db

AkihiroSuda and others added 30 commits August 19, 2019 18:54
integration-on-swarm had unnecessary complexity and was too hard to
maintain. Also, it didn't support the new non-CLI integration test suite.

I'm now doing some experiments out of the repo using Kubernetes:
https://github.com/AkihiroSuda/kube-moby-integration

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e7fbe8e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This removes all the installation steps for docker-py from the
Dockerfile, and instead builds the upstream Dockerfile, and runs
docker-py tests in a container.

To test;

```
make test-docker-py

...

Removing bundles/

---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-dev
Created binary: bundles/dynbinary-daemon/dockerd-dev

---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary docker
Starting dockerd
INFO: Waiting for daemon to start...
.
INFO: Building docker-sdk-python3:3.7.0...
sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402
INFO: Starting docker-py tests...
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0
rootdir: /src, inifile: pytest.ini
plugins: timeout-1.3.3, cov-2.6.1
collected 359 items

tests/integration/api_build_test.py .......s....
....
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7bfe48c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 968345b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ba8f4c7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When building this image docker-in-docker, the DNS in the environment
may not be usable for the build-container, causing resolution to fail:

```
02:35:31 W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg  Temporary failure resolving 'deb.debian.org'
```

This patch detects if we're building from within a container, and if
so, skips creating a networking namespace for the build by using
`--network=host`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c15cea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 980f281)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
See if networking works if we run it first

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6aafe0f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The test-integration/test=integration-cli directory contains
a directory for each daemon that was created during the integration
tests, which makes it a long list to browse through. In addition,
some tests spin up multiple daemons, and when debugging test-failures,
the daemon-logs often have to be looked at together.

This patch organizes the bundl directory to group daemon storage
locationos per test, making it easier to find information about
all the daemons that were used in a specific test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9b5e788)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When connecting with the daemon using a UNIX socket, the HTTP hostname was set, based
on the socket location, which was generating some noise in the test-logs.

Given that the actual hostname is not important (the URL just has to be well-formed),
the hostname/address can be cleaned up to reduce the noise.

This patch strips the path from the `addr`, and keeps `<random-id>.sock` as address.

Before:

    daemon.go:329: [d15d31ba75501] error pinging daemon on start: Get http://%2Ftmp%2Fdocker-integration%2Fd15d31ba75501.sock/_ping: dial unix /tmp/docker-integration/d15d31ba75501.sock: connect: no such file or directory

After:

    daemon.go:329: [d15d31ba75501] error pinging daemon on start: Get http://d15d31ba75501.sock/_ping: dial unix /tmp/docker-integration/d15d31ba75501.sock: connect: no such file or directory

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 92e6e7d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Codecov has shown to be flaky, and calculate the wrong diff, in
addition, it doesn't show coverage for integration tests, which
makes the coverage report not useful.

Removing it for now, while we look at alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bd5c537)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 9d98458)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
instead of vfs

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit ccfaf1e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: zelahi <elahi.zuhayr@gmail.com>
(cherry picked from commit 0ecd6ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit c222c5a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
(cherry picked from commit 790da6c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. Use `go list` to get list of integration dirs to build. This means we
   do not need to have a valid `.go` in every subdirectory and also
   filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
   integration-cli. This is needed so both suites can be run AND set
   custom flags... since the cli suite does not support standard go
   flags.
3. Add options to skip an entire integration suite.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit abece9b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 42f0a0d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
From the code style guidelines;
https://wiki.jenkins.io/display/JENKINS/Code+Style+Guidelines

> 1. Use spaces. Tabs are banned.
> 2. Java blocks are 4 spaces. JavaScript blocks as for Java. XML nesting is 2 spaces

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a95f16c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 722d582)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The main Dockerfile is multi-arch now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 61fd8b7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a28f2a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f648964)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 79713d8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a0bf935)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Container and image names are already unique because they have
the git-sha or build-number, and a single machine won't be running
tests for multiple architectures.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 337d03a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9f0e10f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b04c769)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3897796)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 355bcf6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch removes the manual steps to resolve the Git commit, and
instead, uses the `GIT_COMMIT` that's set by Jenkins's Git plugin.

Behavior changes slightly, because `GIT_PLUGIN` contains the full
commit-sha, not the short one.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit be0e6e9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Tibor Vass and others added 18 commits August 19, 2019 18:58
…3 parallel runs

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit e554fb2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The powerpc-master stage will just run the integration-cli tests. The
existing powerpc stage will run the unit tests and the integration
tests. In this way, PR check jobs will be shorter, but all integration
tests will run after PR is merged to master.

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit c2f9d58)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The z-master stage will just run the integration-cli tests. The
existing z stage will run the unit tests and the integration
tests. In this way, PR check jobs will be shorter, but all
integration tests will run after PR is merged to master.

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit bdc1c1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit bf70a59)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 3564b03)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 4e2f39c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In case a job fails before even generating a report file.

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 0cfc1ec)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit a049ea1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit ad29f9e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit eb30f0a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This will send the results directly after the tests complete,
and make the stage more atomic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7f9328a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Jenkins groups them per stage, so collecting them for all architectures
is possible (without them conflicting or becoming ambiguous)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e2f5b78)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- SC2006: use $(...) notation instead of legacy backticked `...`
- SC2086: double quote to prevent globbing and word splitting

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0b3d201)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b04cbf1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 535e29d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5969bbe)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This also makes sure that we can test all functionality of the
daemon, because some features are not available on static binaries.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4ddb40e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8b6da9d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah thaJeztah added this to the 19.03.2 milestone Aug 19, 2019
@thaJeztah
Copy link
Member Author

ping @andrewhsu @tiborvass @psftw - what's needed to make the new Jenkins run in this repository?

@thaJeztah

This comment has been minimized.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2cffe9b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Attempting to fix;

```
21:16:00 Traceback (most recent call last):
21:16:00 File "/usr/local/bin/yamllint", line 11, in <module>
21:16:00 sys.exit(run())
21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/cli.py", line 170, in run
21:16:00 problems = linter.run(f, conf, filepath)
21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/linter.py", line 233, in run
21:16:00 content = input.read()
21:16:00 File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
21:16:00 return codecs.ascii_decode(input, self.errors)[0]
21:16:00 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 123522: ordinal not in range(128)
21:16:00 Build step 'Execute shell' marked build as failure
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b5e5cac)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Copy link
Member Author

Failure on docker-py; looks either flaky/racy, or something changed;

20:41:13 __________________ AttachContainerTest.test_attach_no_stream ___________________
20:41:13 tests/integration/api_container_test.py:1250: in test_attach_no_stream
20:41:13     assert output == 'hello\n'.encode(encoding='ascii')
20:41:13 E   AssertionError: assert b'' == b'hello\n'
20:41:13 E     Right contains more items, first extra item: 104
20:41:13 E     Use -v to get the full diff

Copy link

@andrewhsu andrewhsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah
Copy link
Member Author

3 LGTM's; I call that a "merge"

@thaJeztah thaJeztah merged commit 9ae801c into docker-archive:19.03 Aug 20, 2019
@thaJeztah thaJeztah deleted the 19.03_backport_jenkinsfile branch August 20, 2019 21:34
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants