[RELAND] .circleci: Improve docker image build workflow #38279
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 7c2853b.
seemethere
changed the title
💊 CI failures summary and remediationsAs of commit 49daeb8 (more details on the Dr. CI page):
1 failure not recognized by patterns:
🚧 3 fixed upstream failures:These were probably caused by upstream breakages that were already fixed.
Please rebase on the
|
ezyang
approved these changes
May 12, 2020
seemethere
changed the title
There was a problem hiding this comment.
@seemethere is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
malfet
reviewed
May 12, 2020
| name: dynamically generate tags to keep | ||
| no_output_timeout: "1h" | ||
| command: | | ||
| GENERATED_IMAGE_TAG=$(\ |
There was a problem hiding this comment.
As @ezyang suggested in original PR, can you add some explanation on what this command is doing?
I.e. generated image tag is SHA sum of last commit that touched .circleci/docker folder.
|
Is there any difference between this PR and #37976 |
This was referenced May 14, 2020
facebook-github-bot
pushed a commit
that referenced
this pull request
May 14, 2020
Summary: Previous attempts to get this right: * #38335 * #38279 * #37976 This tag kept getting deleted before the docker image ci workflow could be merged causing it to have upstream breakages. It'd be best to make sure the garbage collector just doesnt garbage collect it. This is a pre-step to merge #38484 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: #38483 Differential Revision: D21577359 Pulled By: seemethere fbshipit-source-id: c4e0709bd8fff8f24a988b60eaa9f8c01576ef2f
facebook-github-bot
pushed a commit
that referenced
this pull request
May 15, 2020
Summary: closes #37855 Relies on #38483 Previous attempts to get this right: * #38335 * #38279 * #37976 This reverts commit 8063960. Improves the docker image build workflow from many steps to basically transparent from a user's perspective. To update docker images now all one has to do is edit the .circleci/docker folder and it will update automatically and also dynamically add the tags to the list of tags to keep from the garbage collector. Adding a new image will currently stay the same but we can explore doing that dynamically as well. How the build workflow works: - Docker tags are determined by the hash defined from git for the .circleci/docker sub-directory (extracted using git rev-parse) - Images are only built if the computed hash is not found in ecr and the hash is different than the previously computed hash. The previously computed hash is found using the same process as before but subbing out HEAD for the merge base between HEAD and the base git revision - That tag is then passed through the jobs using a shared workspace which is added to downstream jobs using the circleci ${BASH_ENV} How the new garbage collection works: - Tags to keep are generated by stepping through all of the commits in in the .circleci/docker subdirectory Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Pull Request resolved: #38484 Differential Revision: D21585458 Pulled By: seemethere fbshipit-source-id: 37792a1e0f5e5531438c4ae61507639c133aa76d
This was referenced May 18, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This reverts commit 7c2853b.