-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Q: Proper bake usage withh groups #117
Comments
Looking at your use case this is the best option as it will run each target in a dedicated runner. With You're also right about the cache. Better to have it scoped to each target. We use this pattern often across our repos like: https://github.com/moby/buildkit/blob/2816a8328fb13b81bc26b6fd70c67be1ca86e5c6/.github/workflows/validate.yml#L23-L60 We also have a similar pattern but for platforms to build each platform on a dedicated runner to produce binaries: https://github.com/docker/buildx/blob/4a73abfd645457ee8819855cb1a337ab23f833dc/.github/workflows/build.yml#L55-L114 We discussed about this internally few days ago and we were thinking about a simple helper to generate the matrix instead of relying on For grouped targets for example: group "foo" {
targets = ["A", "B", "C"]
} - name: Create matrix
id: targets
use: docker/bake-action/gen-matrix@v2
with:
name: foo
field: targets For platforms or other fields: target "binaries-cross" {
platforms = [
"linux/amd64",
"linux/arm/v7",
"linux/arm64",
"linux/ppc64le",
"linux/riscv64",
"linux/s390x"
]
} - name: Create matrix
id: targets
use: docker/bake-action/gen-matrix@v2
with:
name: foo
field: platforms Let me know what you think. |
Thank you for the analysis. My thoughts were similar although from my limited testing I have not noticed a big difference in the build times between the two options, however A helper would be much appreciated for sure, would isolate possible issues with output formats changing in the future (hopefully). Is that |
Not yet, it's just a suggestion/proposal we could implement.
Yeah that's kinda hard in the same runner, you need to scope each known targets of the group like: - name: Push Images using BuildX Bake
uses: docker/bake-action@v2
with:
files: |
./docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: prod
push: true
set: |
A.cache-from=type=gha,scope=build-prod-A
A.cache-to=type=gha,scope=build-prod-A,mode=max
B.cache-from=type=gha,scope=build-prod-B
B.cache-to=type=gha,scope=build-prod-B,mode=max
C.cache-from=type=gha,scope=build-prod-C
C.cache-to=type=gha,scope=build-prod-C,mode=max I wonder if we could also generate caches inputs if a grouped target is being built. |
Great thank you For the multi-scoped targets, yes I get that but that again requires us to basically make a matrix to know that this needs to be set up. And if we already have a matrix then might as well use the strategy to make it parallel. One more thing, using the matrix strategy does not prevent some images being pushed and others not due to failure. i.e there is no sync feature to make sure that all targets succeed. Currently the workaround is to do it twice, once for building without push, and the second with push enabled (and hopefully cache is reused). Do you think that would ever change with the use case that a the target is just a group? Would that automatically sync the images? |
Hi, I am just wondering what is the proper/best practice for using this action with multiple targets through a group.
As an example I have a repository that contains 3 subsystems that are controlled through a bakefile that trigger them.
Example
I want to trigger this group through the action, but am unsure as to the best approach. I have the following two options but unsure which way is best, or if there is another option I have not considered.
NOTE: removed the surrounding boiler stuff such as credential logins, QEMU/BuildX setup etc.
Simply use the
prod
as thetargets
input into the action. This will trigger the build of all of them however does this consider:a) multi-platform builds
b) proper caching using gha, since the group is used do they share the cache or do they overwrite each others?
Example
Currently I am taking the group input, and using jq to extract the targets within it and generating a matrix strategy so that they are run in parallel and guarantee separate caches. This makes things a bit more complicated and a little harder to maintain (for instance already had an issue with the bake output changing on me and needing to update the
jq
extraction.Then using that to build the matrix
And finally building the images of those targets
The text was updated successfully, but these errors were encountered: