buildx multi-platform docker builds fail

[dustinrue]

Definitely not sure if this would go here or in lima but I'm unable to do multi-arch builds using buildx. Here is an example error that I get

docker buildx build . -t base-php --platform linux/amd64,linux/arm64
...
#12 36.47 E: Sub-process /usr/bin/dpkg returned an error code (1)
------
Dockerfile:9
--------------------
   8 |
   9 | >>> RUN \
  10 | >>>   apt update && \
  11 | >>>   apt dist-upgrade -y && \
  12 | >>>   apt install gnupg -y && \
  13 | >>>   apt clean
  14 |
--------------------
error: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/bash -c apt update &&   apt dist-upgrade -y &&   apt install gnupg -y &&   apt clean" did not complete successfully: exit code: 100

I get this same error on both my 2018 i7 Mac mini and my 2020 M1 MBA.

This is my config

vm:
    cpu: 6
    disk: 60
    memory: 6
    mounts:
        - /Users/dustin/wp-local-docker-sites:w
        - /Users/dustin/.wplocaldocker:w
runtime: docker
kubernetes:
    enabled: false
    version: v1.22.2

[abiosoft]

What is the expected behaviour? Does this work out of the box with docker for mac?

I was able to get it to run successfully by having multiple nodes as per the instruction on https://docs.docker.com/buildx/working-with-buildx/.

Steps

Note:
The following commands are not available until the next release of Colima, which supports multiple instances and docker contexts. For now you can build from source or brew install --head colima

1. Create instances for the two architectures.

The instance with non-native architecture will run noticeably slower. This is expected as the processor is being emulated by qemu.

Create an instance for amd64/x86_64 architecture

colima start --profile amd --arch amd

Create an instance for arm64/aarch64 architecture

colima start --profile arm --arch arm

2. Create a buildx context to use the created instances as nodes.

Step 1 (if successful) should create colima-amd and colima-arm docker contexts. Specify them in the new buildx context.

docker buildx create --use --name custom colima-amd
docker buildx create --append --name custom colima-arm

3. Run docker buildx build

Everything should be fine now and running docker buildx build should work as desired.

[abiosoft]

I will be closing as this is not really a Colima issue.

[dustinrue]

Thanks for taking a look! I gave this a try and it seems to be working well.

FWIW it does in fact work with 0 modifications when using Docker Desktop. Using this as a solution is perfectly fine though.

[abiosoft]

If that is the case, Docker Desktop handles it internally for better user experience.

I will revisit this later.

[abiosoft]

I have found a proper solution to this that would require almost no setup from the user :)
Will push a fix soon.

[dustinrue]

Excellent! I look forward to testing it. FWIW I am having good luck with Colima on my Intel Mac mini and my M1 powered MBA.

[abiosoft]

Though this is still a valid approach #44 (comment), multiple instances are no longer required. You can reverse those steps.

Kindly update to the latest development version brew install --head colima and start colima (or restart if running). Multi-architecture support should now be there out of the box.

Running images of different architectures should work fine. e.g.

docker run --rm --platform linux/amd64 alpine
docker run --rm --platform linux/aarch64 alpine 

Using docker buildx build requires at most one step.

# this is assuming default profile, 
# otherwise the name would be `colima-<profile>` e.g. `colima-arm`.
docker buildx create --use colima

[dustinrue]

Confirmed, this is working perfectly for me and I am able to build and run multi-arch images just like before! Nice work!

[gayanhewa]

Any idea what would cause this on the m1?

[goshlanguage]

Any idea what would cause this on the m1?

@gayanhewa m1s use arm64 architecture, not amd64 as shown in your screenshot. Your CPU doesn't know how to talk amd64 so when you try to run that specifically, it can't, resulting in exec format error

[abiosoft]

@gayanhewa @goshlanguage this should no longer happen as cross-architecture is supported (though limited).

More complex images like java jvm or mssql-server may not behave correctly but most of the amd64 images will work fine on M1.

Nonetheless, it is better and recommended to use the arm64 images instead.