Skip to content

[RELAND] .circleci: Improve docker image build workflow #38484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Parent: Add extra debug info for the super common "index out of bounds" error.
wants to merge 1 commit into from
Closed

[RELAND] .circleci: Improve docker image build workflow #38484

wants to merge 1 commit into from

Conversation

@seemethere
Copy link
Member

@seemethere seemethere commented May 14, 2020 •
edited
Loading

closes #37855

Relies on #38483

Previous attempts to get this right:

This reverts commit 8063960.

Improves the docker image build workflow from many steps to basically
transparent from a user's perspective.

To update docker images now all one has to do is edit the
.circleci/docker folder and it will update automatically and also
dynamically add the tags to the list of tags to keep from the garbage
collector.

Adding a new image will currently stay the same but we can explore doing
that dynamically as well.

How the build workflow works:

  • Docker tags are determined by the hash defined from git for the
    .circleci/docker sub-directory (extracted using git rev-parse)
  • Images are only built if the computed hash is not found in ecr and
    the hash is different than the previously computed hash. The
    previously computed hash is found using the same process as before
    but subbing out HEAD for the merge base between HEAD and the base
    git revision
  • That tag is then passed through the jobs using a shared workspace
    which is added to downstream jobs using the circleci ${BASH_ENV}

How the new garbage collection works:

  • Tags to keep are generated by stepping through all of the commits in
    in the .circleci/docker subdirectory

Signed-off-by: Eli Uriegas eliuriegas@fb.com

@seemethere seemethere marked this pull request as draft May 14, 2020 16:38
@seemethere seemethere mentioned this pull request May 14, 2020
Closed
@dr-ci
Copy link

dr-ci bot commented May 14, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit 7e32363 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 17 times.

@seemethere seemethere force-pushed the ci-all/reland_reland_reland_improve_docker_ci_workflow branch 2 times, most recently from 4d9dc3c to 5b62f34 Compare May 14, 2020 18:02
facebook-github-bot pushed a commit that referenced this pull request May 14, 2020
@seemethere @facebook-github-bot
.cirlceci: Keep tags that look like a sha1 (#38483)
3300dd5 
Summary:
Previous attempts to get this right:
* #38335
* #38279
* #37976

This tag kept getting deleted before the docker image ci workflow could
be merged causing it to have upstream breakages.

It'd be best to make sure the garbage collector just doesnt garbage
collect it.

This is a pre-step to merge #38484

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Pull Request resolved: #38483

Differential Revision: D21577359

Pulled By: seemethere

fbshipit-source-id: c4e0709bd8fff8f24a988b60eaa9f8c01576ef2f
@seemethere seemethere force-pushed the ci-all/reland_reland_reland_improve_docker_ci_workflow branch 2 times, most recently from cd2909f to 24a5236 Compare May 14, 2020 21:26
@seemethere seemethere marked this pull request as ready for review May 14, 2020 21:52
@seemethere seemethere requested review from ezyang and malfet May 14, 2020 21:52
@seemethere seemethere force-pushed the ci-all/reland_reland_reland_improve_docker_ci_workflow branch 3 times, most recently from 413950c to 15e9833 Compare May 14, 2020 22:16
@seemethere
.circleci: Improve docker image build workflow
7e32363 
This reverts commit 8063960.

Improves the docker image build workflow from many steps to basically
transparent from a user's perspective.

To update docker images now all one has to do is edit the
.circleci/docker folder and it will update automatically and also
dynamically add the tags to the list of tags to keep from the garbage
collector.

Adding a new image will currently stay the same but we can explore doing
that dynamically as well.

How the build workflow works:
  - Docker tags are determined by the hash defined from git for the
    .circleci/docker sub-directory (extracted using git rev-parse)
  - Images are only built if the computed hash is not found in ecr and
    the hash is different than the previously computed hash. The
    previously computed hash is found using the same process as before
    but subbing out HEAD for the merge base between HEAD and the base
    git revision
  - That tag is then passed through the jobs using a shared workspace
    which is added to downstream jobs using the circleci ${BASH_ENV}

How the new garbage collection works:
  - Tags to keep are generated by stepping through all of the commits in
    in the .circleci/docker subdirectory

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
@seemethere seemethere changed the title .circleci: Improve docker image build workflow [RELAND] .circleci: Improve docker image build workflow May 14, 2020
@seemethere seemethere added the module: ci Related to continuous integration label May 14, 2020
@seemethere seemethere force-pushed the ci-all/reland_reland_reland_improve_docker_ci_workflow branch from 15e9833 to 7e32363 Compare May 14, 2020 22:44
malfet
malfet approved these changes May 14, 2020
View reviewed changes
Copy link
Contributor

@malfet malfet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's hope it will be green this time

facebook-github-bot
facebook-github-bot reviewed May 14, 2020
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seemethere is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ailzhang
Copy link
Contributor

ailzhang commented May 15, 2020 •
edited
Loading

Sorry I had to revert this PR since it broke pytorch/xla CI. (not the xla job in our CI, but the CI in pytorch/xla repo).
XLA CI is parsing our config.yml file to make sure we always use the same docker image as pytorch/pytorch repo. Since this PR moves the tag into a BASH_ENV, we'll need that to be setup for pytorch/xla as well.
https://github.com/pytorch/xla/blob/master/.circleci/parse_docker_image.py
Happy to sync more offline to get this relanded. Sorry about this!

This was referenced May 18, 2020
Closed
Closed
@facebook-github-bot facebook-github-bot deleted the ci-all/reland_reland_reland_improve_docker_ci_workflow branch July 13, 2020 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@malfet malfet malfet approved these changes

@ezyang ezyang Awaiting requested review from ezyang

Assignees

No one assigned

Labels

module: ci Related to continuous integration

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[RFC] Improve the CI docker image builder workflow

4 participants

@seemethere @ailzhang @malfet @facebook-github-bot