Run build steps for multi-arch sequentially #7333
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We saw that if we try to build using an ENV var containing all the architectures then docker buildx will run 3 parallel jobs. This made the GH runners crash in some cases because they were over-utilizing the resources. To mitigate this we'll sequentially build each image for each arch. Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com>
thesuperzapper
requested changes
Oct 12, 2023
There was a problem hiding this comment.
@kimwnasptd can I request we do the amd64 builds first, so they fail faster for broken images?
Also, can we make the same re-ordering change on the other workflows (I am happy to do this in a separate PR, if you want).
|
@thesuperzapper I'm on it |
Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com>
|
@thesuperzapper just pushed a commit, PTAL |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: thesuperzapper The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
kimwnasptd
added a commit
to kimwnasptd/kubeflow
that referenced
this pull request
Oct 13, 2023
* Run build steps for multi-arch sequentially We saw that if we try to build using an ENV var containing all the architectures then docker buildx will run 3 parallel jobs. This made the GH runners crash in some cases because they were over-utilizing the resources. To mitigate this we'll sequentially build each image for each arch. Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * review: Build first for amd64 Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> --------- Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com>
kimwnasptd
added a commit
to kimwnasptd/kubeflow
that referenced
this pull request
Oct 13, 2023
* Run build steps for multi-arch sequentially We saw that if we try to build using an ENV var containing all the architectures then docker buildx will run 3 parallel jobs. This made the GH runners crash in some cases because they were over-utilizing the resources. To mitigate this we'll sequentially build each image for each arch. Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * review: Build first for amd64 Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> --------- Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com>
This was referenced Oct 13, 2023
google-oss-prow bot
pushed a commit
that referenced
this pull request
Oct 16, 2023
* ci: Build ARM images for core components (#7220) * ci: Build ARM images for core components Extend the GH Actions to also build the images on ARM architectures. Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * crud-web-apps: Update python and gevent versions In order to successfully build on linux/arm64/v8 we'll need to: * Update to Python 3.10 * Bump the gevent version gevent/gevent#1721 (comment) * Update the workflows for JWA Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for centraldb Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for kfam Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for notebook-controller Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for PodDefaults Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for Profile Controller Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for pvcviewer controller Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for TensorBoard Controller Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for TWA Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update workflows for VWA Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Update releasing script to include PVCViewers Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> --------- Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * Run build steps for multi-arch sequentially (#7333) * Run build steps for multi-arch sequentially We saw that if we try to build using an ENV var containing all the architectures then docker buildx will run 3 parallel jobs. This made the GH runners crash in some cases because they were over-utilizing the resources. To mitigate this we'll sequentially build each image for each arch. Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * review: Build first for amd64 Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> --------- Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * ci: Ensure we publish for all architectures (#7343) Previously we omitted the ARCH env var, which resulted in the last step to not push for all architectures, but only for amd64. This commit will set a default ENV var with all architectures, which will be picked up by the build-push make rule we have. Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> * ci: Push the multi-arch image for specific tag (#7345) The latest changes made the CI to only build the images for different archs, but not push for the tag value of the commit. While next steps, for pushing the latest tag or a release version tag will work we still want to ensure we push an image with a tag. Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com> --------- Signed-off-by: Kimonas Sotirchos <kimwnasptd@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We saw that if we try to build using an ENV var containing all the architectures then docker buildx will run 3 parallel jobs.
This made the GH runners crash in some cases because they were over-utilizing the resources.
To mitigate this we'll sequentially build each image for each arch.
/assign @thesuperzapper
cc @DnPlas