Support bake file as build target

[jalaziz]

Description

Today, a docker compose file can service as a bake file. This is great when you already use a docker compose file for your build spec, but docker bake allows for much more robust and complex build configurations than what compose supports.

Now that Bake is GA, it would be great if we could instead specify a bake file and target as the build spec for a compose service. This would allow for much richer build definitions that are shared across development and production, while still allowing existing docker compose workflows to run as is.

Put another way, it would be wonderful if docker compose up --build could invoke a Bake build rather than only being able to reference a standard Dockerfile.

[ndeloof]

What do you have in mind as "more robust and complex build configurations" when it comes to a single image being built ?
AFAICT docker compose build definition gives you all the features bake supports, and actually latest Docker Compose release can rely on bake under the hood to run the build, which make multi-services sharing common resources build more efficient.

[jalaziz]

What do you have in mind as "more robust and complex build configurations" when it comes to a single image being built ?

Yes. Perhaps the most powerful feature of bake is being able to coordinate builds across dependencies. For example, with Bake, it's trivial to have a base image that is used in your final images and have it built automatically as needed. It's also trivial to use inheritance and targets to share a Dockerfile between a production build and debug-enabled dev build. Finally, Docker Bake supports functions.

AFAIK, none of what I've listed is supported by the docker compose build configuration without an extra layer of coordination.

[hanikesn]

I'd like to add that I also use matrices to easily define dozens of targets for microservices sharing a common go build step including cross platform builds and properly configured caching. Bake files enable many advanced use cases for mono repos. Being able to reuse those targets from a docker-compose filed would be immensely helpful.

[ndeloof]

I can hardly imagine how compose could run bake build while still managing multiple services declared in compose.yaml - especially as our middle terms plan is to remove native build support from compose and delegate builds to bake, generating a bake.json config from compose declaration.

The only option I can imagine is we support #12376

IMHO if you heavily rely on a bakefile to optimize your build, you should just run docker buildx bake before you run compose commands.

Closing as "not planned"

[hanikesn]

IMHO if you heavily rely on a bakefile to optimize your build, you should just run docker buildx bake before you run compose commands.

The nice part of having compose itself support it is the integration in 3rd party tooling, e.g. launching a devcontainer or similar. That way you wouldn't have to instruct users and it'd be possible to use them in Github Codespaces as well.

#12376 would take care of that as well of course.

Thanks for your consideration!

[antiBaconMachine]

AFAICT docker compose build definition gives you all the features bake supports

It mostly does this if you are prepared to replicate your entire existing bake target into the compose file and flatten any inheritence whilst your are at it.

I'm not actually sure I can see a way to do build dependencies if you did this either e.g. have one target depend on a base image build.

IMO docker compose is a great general purpose orchestration system, bake is a great build tool. I think the more specific tool should be consumed by the more general tool.

Right now it works the other way around, bake can consume compose files and generate targets from them. Given the bake definitions in compose files are somewhat more limited than a full external bake definition (no inheritence, have to flatten and duplicate everywhere) this all seems suboptimal.

I get that compose has always been a fully declarative source of truth, it feels odd to delegate some of it's work to a completely different declarative system and there a probably a hundred other issues I haven't thought about.

Personal data point from my experience: The way we use these systems is

• Bake is the source of truth for production images
• Compose is used for developer orchestration

So currently we end up with developer images which are defined similarly in x-bake sections but not quite the same as our actual production images in a docker-bake.hcl file. It looks very attractive to us to have docker compose use the existing bake targets from our bake files, not have to specify them again in compose.

[cideM]

I'd like to add a somewhat concrete use case for this: for local development we use the --mount=type=ssh feature so that docker build can leverage the host's SSH agent to e.g., authenticate with private GitHub repositories. This is setup now so that docker buildx bake Just Works™ (and it was a lot of fun to setup -- I really like Bake).

We also have a Docker Compose file that starts the service I alluded to above. I expected that docker compose up --build would utilize the Bake configuration and therefore use SSH agent forwarding. But the build started with docker compose up --build fails with an SSH authentication error, whereas docker buildx bake works.

[ndeloof]

@cideM see https://github.com/compose-spec/compose-spec/blob/main/build.md#ssh

[cideM]

I know about that and am using it but now I’m duplicating this exact same configuration between Bake and Compose

[ndeloof]

Let's keep this one opened to collect feedback and integration proposals