Unable to build multi-platform images without also pushing them

[natescherer]

Hello! Looks like when the --no-push parameter on devcontainer build was dropped it stopped being possible to build a multi-platform container without pushing it. The current code sets the behavior when you don't push to automatically add --load to the docker buildx build call, which is incompatible with manifest based Docker images (which multi-platform images must be).

I will cook up a PR to fix this.

[Chuxel]

Nate Scherer (@natescherer) With buildx, you can specify multiple platforms to build in a single call. Are you looking to manually construct the manifest? (Or put another way, given you can't --load a multi-platform build, what would you do with it beyond --push unless you are creating the manifest manually?)

[natescherer]

So what I'm trying to do is build multi-platform image support into https://github.com/devcontainers/ci. Currently that's set up with three stages, build, run an (optional) command inside the image, and then push. I was trying to maintain the same workflow for multi-container images. Current thinking is adding --output-type and --output-path options to the CLI that map to the docker buildx --output, which would let me dump the container in OCI format out, then push it to GHCR.

Does that make sense?

Also I now believe I was incorrect in my first post, I am not sure building an image without pushing ever worked previously, I was just reading logs wrong.

[Chuxel]

Given the dev container CLI supports --platform which like buildx supports a list, so wouldn't wiring that up be a bit easier?

e.g., this should work: devcontainer build --platform "linux/amd64,linux/arm64"

EDIT: That said, the --help doesn't say that, so that bit should be tweaked. It's the same syntax as buildx and I always found that it was --platform instead of --platforms (w/the s) strange.

[natescherer]

Oh yeah, that's what I'm doing. But given the only way that call works now is with --push, there's no way to run the intermediate command (unless I'm mistaken). That's what I'm trying to preserve: the ability to build, test, and then push that's currently in the Action.

[Chuxel]

Do you mean the runCmd? Yeah, that would require --load unfortunately since it ultimately is a "docker run + exec". So you'd end up having to do an image pull or something after it was created anyway - another single architecture build that hopefully reuses the cache. So you're thinking that having an output type would allow you to export and then import and push?

[natescherer]

Yeah, runCmd is what I was talking about, sorry for not specifying. That part actually still works, because the devcontainer up that runCmd does recognizes it doesn't have a local image it can use an builds it again just for the local platform. Not super efficient, obviously but gets the job done. (If you want to see for yourself, check this out: https://github.com/natescherer/sre-devcontainer/runs/8139243944?check_suite_focus=true)

I'm still trying to figure out how to push the OCI tarball to GHCR, but, if I can figure that out, I do think that adding output options to the CLI would solve the problem.

[natescherer]

Okay, after a LOT of testing, I've determined that you can run this to copy a multi-image OCI tarball to GHCR successfully: skopeo copy --all oci-archive:multi-platform-image.tar docker://ghcr.io/owner/repo:latest

I made quick and dirty edits to the CLI to tack --output type=oci,dest=multi-platform-image.tar onto the buildx call while I was proving this out, I will expand that out to add proper --output-type and --output-dest options to the CLI, add doc and tests, and make a PR!

[natescherer]

Resolved by #166