Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[v1.9.x] Backport docker cache fixes#20537

Merged
josephevans merged 3 commits intoapache:v1.9.xfrom
josephevans:docker_cache_fixes_v1.9.x
Aug 19, 2021
Merged

[v1.9.x] Backport docker cache fixes#20537
josephevans merged 3 commits intoapache:v1.9.xfrom
josephevans:docker_cache_fixes_v1.9.x

Conversation

@josephevans
Copy link
Copy Markdown
Contributor

This is not required for the v1.9 release, but it should speed up the CI if we have additional PRs for the v1.9.x branch.

Backport #20385, #20458 and #20525.

* libturbojpeg0-dev is only for ubuntu 18.04, so don't install on other ubuntu versions.

* Change logic to not install on 16.04 (works on 18.04 and 20.04.)

* Fix docker registry variable - was inadvertently changed to DOCKER_CACHE_REGISTRY from DOCKER_ECR_REGISTRY.

* Use our local ECR registry for docker-compose images as well.

* Revert changing DOCKER_CACHE_REGISTRY variable.

* Override DOCKER_CACHE_REGISTRY for docker-compose in docker cache pipeline.

* Login to ECR if using ECR for docker registry.

* Import docker_cache when needed.
* Properly initialize set so that docker cache builds exclude the centos7 dockerfile.

* Change the number of parallel builds to 1 for easier debugging and bandwidth control, also parameterize the number of build retries and reduce from 10 to 1 for easier debugging.

* Remove old dockerfiles.

* Update docker-compose config to use new ECR-based naming scheme.

* Refactor build.py and docker_cache.py to support building and pushing containers for the docker-compose based platforms. Add options for selecting architecture (defaults to x86) since aarch64 container builds will fail on x86 systems.

* Remove old commands for building docker-compose based containers since they are not built in docker_cache.py.

* Fix running of docker-compose based containers via build.py.
@mxnet-bot
Copy link
Copy Markdown

Hey @josephevans , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [windows-cpu, windows-gpu, miscellaneous, website, centos-gpu, centos-cpu, unix-cpu, clang, unix-gpu, edge, sanity]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress pr-awaiting-review PR is waiting for code review and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels Aug 18, 2021
Copy link
Copy Markdown
Contributor

@waytrue17 waytrue17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@josephevans josephevans merged commit 82384cb into apache:v1.9.x Aug 19, 2021
@josephevans josephevans deleted the docker_cache_fixes_v1.9.x branch August 19, 2021 18:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

pr-awaiting-review PR is waiting for code review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants