Enable pushing early image cache to ghcr.io registry#26119
Merged
potiuk merged 1 commit intoapache:mainfrom Sep 1, 2022
Merged
Enable pushing early image cache to ghcr.io registry#26119potiuk merged 1 commit intoapache:mainfrom
potiuk merged 1 commit intoapache:mainfrom
Conversation
221c8ab to
731c39c
Compare
731c39c to
3e72ab9
Compare
Member
Author
|
This is a reasult of discussion with @o-nikolas and @mik-laj on Slack: https://apache-airflow.slack.com/archives/C015SLQF059/p1661905832810429 This change should make our cache of images much "fresher" even in case our main starts failing. It does not attempt to upgrade constraints - it will try to build the image with current constraints and push it as cache. This is done early in the build workflow and it will not wait for all tests to finish (like the final cache pushing which also upgrades constraints). Result of this change will be that even if tests fail, if the images can be build and succeed, they will be pushed as cache quickly - and this will make subsequent builds to be much faster. |
6c671af to
c011ad2
Compare
In order to react quicker to changes in setup.py, setup.cfg and Dockerfiles, we can push an early image cache to the ghcr.io cache. This build is done way before constraints are regenerated so it is using the "current" constraints at the moment the merge happens. This should be safe - if the image fails to build (we ignore that) the cache will not be pushed, but if it will, then even if we merge a new commit quickly and the rest of the build is cancelled, the cache will be refreshed with new Dockerfile/setup.py/setup.cfg and the subsequent builds will run much faster.
c011ad2 to
fd0e518
Compare
kaxil
approved these changes
Sep 1, 2022
Contributor
|
Thanks for the quick turnaround! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In order to react quicker to changes in setup.py, setup.cfg and
Dockerfiles, we can push an early image cache to the ghcr.io cache.
This build is done way before constraints are regenerated so it is
using the "current" constraints at the moment the merge happens.
This should be safe - if the image fails to build (we ignore that)
the cache will not be pushed, but if it will, then even if we merge
a new commit quickly and the rest of the build is cancelled, the
cache will be refreshed with new Dockerfile/setup.py/setup.cfg and
the subsequent builds will run much faster.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.