This repository has been archived by the owner on Oct 13, 2023. It is now read-only.
forked from moby/moby
-
Notifications
You must be signed in to change notification settings - Fork 425
[19.03 backport] Jenkinsfile and related test-changes #328
Merged
thaJeztah
merged 71 commits into
docker-archive:19.03
from
thaJeztah:19.03_backport_jenkinsfile
Aug 20, 2019
Merged
[19.03 backport] Jenkinsfile and related test-changes #328
thaJeztah
merged 71 commits into
docker-archive:19.03
from
thaJeztah:19.03_backport_jenkinsfile
Aug 20, 2019
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
integration-on-swarm had unnecessary complexity and was too hard to maintain. Also, it didn't support the new non-CLI integration test suite. I'm now doing some experiments out of the repo using Kubernetes: https://github.com/AkihiroSuda/kube-moby-integration Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp> (cherry picked from commit e7fbe8e) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This removes all the installation steps for docker-py from the Dockerfile, and instead builds the upstream Dockerfile, and runs docker-py tests in a container. To test; ``` make test-docker-py ... Removing bundles/ ---> Making bundle: dynbinary (in bundles/dynbinary) Building: bundles/dynbinary-daemon/dockerd-dev Created binary: bundles/dynbinary-daemon/dockerd-dev ---> Making bundle: test-docker-py (in bundles/test-docker-py) ---> Making bundle: .integration-daemon-start (in bundles/test-docker-py) Using test binary docker Starting dockerd INFO: Waiting for daemon to start... . INFO: Building docker-sdk-python3:3.7.0... sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402 INFO: Starting docker-py tests... ============================= test session starts ============================== platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0 rootdir: /src, inifile: pytest.ini plugins: timeout-1.3.3, cov-2.6.1 collected 359 items tests/integration/api_build_test.py .......s.... .... ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 7bfe48c) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 968345b) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit ba8f4c7) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When building this image docker-in-docker, the DNS in the environment may not be usable for the build-container, causing resolution to fail: ``` 02:35:31 W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg Temporary failure resolving 'deb.debian.org' ``` This patch detects if we're building from within a container, and if so, skips creating a networking namespace for the build by using `--network=host`. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 3c15cea) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 980f281) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
See if networking works if we run it first Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 6aafe0f) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The test-integration/test=integration-cli directory contains a directory for each daemon that was created during the integration tests, which makes it a long list to browse through. In addition, some tests spin up multiple daemons, and when debugging test-failures, the daemon-logs often have to be looked at together. This patch organizes the bundl directory to group daemon storage locationos per test, making it easier to find information about all the daemons that were used in a specific test. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 9b5e788) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When connecting with the daemon using a UNIX socket, the HTTP hostname was set, based on the socket location, which was generating some noise in the test-logs. Given that the actual hostname is not important (the URL just has to be well-formed), the hostname/address can be cleaned up to reduce the noise. This patch strips the path from the `addr`, and keeps `<random-id>.sock` as address. Before: daemon.go:329: [d15d31ba75501] error pinging daemon on start: Get http://%2Ftmp%2Fdocker-integration%2Fd15d31ba75501.sock/_ping: dial unix /tmp/docker-integration/d15d31ba75501.sock: connect: no such file or directory After: daemon.go:329: [d15d31ba75501] error pinging daemon on start: Get http://d15d31ba75501.sock/_ping: dial unix /tmp/docker-integration/d15d31ba75501.sock: connect: no such file or directory Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 92e6e7d) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Codecov has shown to be flaky, and calculate the wrong diff, in addition, it doesn't show coverage for integration tests, which makes the coverage report not useful. Removing it for now, while we look at alternatives. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit bd5c537) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit 9d98458) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
instead of vfs Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit ccfaf1e) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: zelahi <elahi.zuhayr@gmail.com> (cherry picked from commit 0ecd6ab) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> Signed-off-by: Tibor Vass <tibor@docker.com> (cherry picked from commit c222c5a) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Michael Zhao <michael.zhao@arm.com> (cherry picked from commit 790da6c) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. Use `go list` to get list of integration dirs to build. This means we do not need to have a valid `.go` in every subdirectory and also filters out other dirs like "bundles" which may have been created. 2. Add option to specify custom flags for integration and integration-cli. This is needed so both suites can be run AND set custom flags... since the cli suite does not support standard go flags. 3. Add options to skip an entire integration suite. Signed-off-by: Brian Goff <cpuguy83@gmail.com> (cherry picked from commit abece9b) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit 42f0a0d) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
From the code style guidelines; https://wiki.jenkins.io/display/JENKINS/Code+Style+Guidelines > 1. Use spaces. Tabs are banned. > 2. Java blocks are 4 spaces. JavaScript blocks as for Java. XML nesting is 2 spaces Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit a95f16c) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 722d582) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The main Dockerfile is multi-arch now. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 61fd8b7) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit a28f2a2) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit f648964) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 79713d8) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit a0bf935) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Container and image names are already unique because they have the git-sha or build-number, and a single machine won't be running tests for multiple architectures. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 337d03a) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 9f0e10f) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit b04c769) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 3897796) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 355bcf6) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch removes the manual steps to resolve the Git commit, and instead, uses the `GIT_COMMIT` that's set by Jenkins's Git plugin. Behavior changes slightly, because `GIT_PLUGIN` contains the full commit-sha, not the short one. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit be0e6e9) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
…3 parallel runs Signed-off-by: Tibor Vass <tibor@docker.com> (cherry picked from commit e554fb2) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The powerpc-master stage will just run the integration-cli tests. The existing powerpc stage will run the unit tests and the integration tests. In this way, PR check jobs will be shorter, but all integration tests will run after PR is merged to master. Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit c2f9d58) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The z-master stage will just run the integration-cli tests. The existing z stage will run the unit tests and the integration tests. In this way, PR check jobs will be shorter, but all integration tests will run after PR is merged to master. Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit bdc1c1a) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit bf70a59) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit 3564b03) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit 4e2f39c) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In case a job fails before even generating a report file. Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit 0cfc1ec) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit a049ea1) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit ad29f9e) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Andrew Hsu <andrewhsu@docker.com> (cherry picked from commit eb30f0a) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This will send the results directly after the tests complete, and make the stage more atomic. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 7f9328a) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Jenkins groups them per stage, so collecting them for all architectures is possible (without them conflicting or becoming ambiguous) Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit e2f5b78) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- SC2006: use $(...) notation instead of legacy backticked `...` - SC2086: double quote to prevent globbing and word splitting Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 0b3d201) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit b04cbf1) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 535e29d) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 5969bbe) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This also makes sure that we can test all functionality of the daemon, because some features are not available on static binaries. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 4ddb40e) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 8b6da9d) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
ping @andrewhsu @tiborvass @psftw - what's needed to make the new Jenkins run in this repository? |
This comment has been minimized.
This comment has been minimized.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 2cffe9b) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Attempting to fix; ``` 21:16:00 Traceback (most recent call last): 21:16:00 File "/usr/local/bin/yamllint", line 11, in <module> 21:16:00 sys.exit(run()) 21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/cli.py", line 170, in run 21:16:00 problems = linter.run(f, conf, filepath) 21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/linter.py", line 233, in run 21:16:00 content = input.read() 21:16:00 File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode 21:16:00 return codecs.ascii_decode(input, self.errors)[0] 21:16:00 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 123522: ordinal not in range(128) 21:16:00 Build step 'Execute shell' marked build as failure ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit b5e5cac) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Failure on docker-py; looks either flaky/racy, or something changed;
|
andrewhsu
approved these changes
Aug 20, 2019
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
tiborvass
approved these changes
Aug 20, 2019
seemethere
approved these changes
Aug 20, 2019
3 LGTM's; I call that a "merge" |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports of:
No conflicts