Enable use of BuildKit in CI

[lbussell]

We want to be able to build Dockerfiles and samples with newer Docker functionality (multi-platform builds, etc). See dotnet/dotnet-docker#4742.

We previously tried to enable BuildKit in ImageBuilder, but encountered an issue where multi-stage Dockerfiles will lose their variant (e.g. arm64v8 becomes arm64), likely due to us building all arm32v7 and arm64v8 Dockerfiles on an arm64 host, and not specifying the build target for the final stage in the Dockerfile build. See #1132 and #1136.

Now we should investigate whether or not we can install and use buildx on our build agents to resolve this issue.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[lbussell]

Related:

• Support scratch stages on build machines where BuildKit disabled #1017

This is the same issue, but what I have observed is the opposite behavior. @mthalman mentions in dotnet/dotnet-docker#4438 (comment) that the issue might have been resolved due to build agent updates, so that might have changed the behavior here.

[lbussell]

@richlander since you are the resident expert on multi-platform Docker builds, can you provide some insight here? If having no --platform= specified in the FROM instruction implies that $TARGETPLATFORM will be the default platform for the final stage, why would the output of the image be different from what's passed in the --platform argument to docker build? Could this be a consequence of not using buildx? We are not yet trying to build multiple platforms at once, just one at a time.

I am still having troubles getting buildx working on Azure build agents. Docker has a "setup buildx" GitHub Action, but GitHub Actions runners already have buildx installed.

[richlander]

Let's assume that $BUILDPLATFORM is not used in the Dockerfile to make this discussion simple.

The value of --platform (via the CLI) will be used for two things:

• The platform to use to select an image from any manifest tag.
• The platform to write to the final image, even if it is wrong.

Here's a good example.

$ echo FROM arm32v7/alpine > Dockerfile
$ docker build -t test --platform linux/amd64 .
$ docker inspect test | grep Arch
        "Architecture": "amd64",
$ docker run --rm test uname -a
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Linux 82793be52fae 5.15.49-linuxkit-pr #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023 armv7l Linux