On apple M1, docker-compose does not honor the platform: tag for amd64 if only the arm64 version of the same image is present

[pcallahan-r7]

Description

If an arm64 version of an image is present but not the amd64 version, the platform tag when set to amd64 in docker-compose is not honored. If both or neither are present, then platform is honored.

Steps to reproduce the issue:

1. Create a test service in a docker-compose file with the linux/amd64 architecture platform specified.

version: '3'
services:

  my-test-service:
    platform: linux/amd64
    image: openjdk:11

Ensure the image does not exist in docker. Then pull the image for the arm64 architecture (but not the amd64 image):

$ docker pull --platform linux/arm64 openjdk:11

Run the test service with uname and observe the wrong architecture:

$ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 docker-compose run --rm   my-test-service  uname -a

Linux ... aarch64 GNU/Linux

This only appears to happen if the arm64 image is already there and the amd64 image is not. If there is no image, docker-compose will pull the correct amd64 image.

Describe the results you received:
docker-compose runs the arm64 image

Describe the results you expected:
docker-compose runs the amd64 image.

Output of docker compose version:

Docker Compose version v2.7.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)
  compose: Docker Compose (Docker Inc., v2.7.0)
  extension: Manages Docker extensions (Docker Inc., v0.2.8)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 11
  Running: 0
  Paused: 0
  Stopped: 11
 Images: 32
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.104-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 5
 Total Memory: 15.6GiB
 Name: docker-desktop
 ID: XA6U:B2LX:OK77:JHUI:PGJX:EC7I:TH26:F2UH:LNCP:JFYI:N2TY:7KJ5
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

[nicksieger]

We made some changes in v2.11 to help with multiple platforms. As of 2.11.2, when I run your example, I get the following error message. Is that more helpful for your use case? Compose doesn't transparently pull the correct platform, but at least alerts you to the mismatch.

$ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 docker-compose run --rm   my-test-service  uname -a
[+] Running 1/0
 ⠿ Network 9740_default  Created                                                                   0.0s
Error response from daemon: image with reference openjdk:11 was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64/v8

If I then pull the correct image locally, uname outputs the correct platform.

$ docker pull --platform linux/amd64 openjdk:11
11: Pulling from library/openjdk
001c52e26ad5: Pull complete
d9d4b9b6e964: Pull complete
2068746827ec: Pull complete
9daef329d350: Pull complete
d85151f15b66: Pull complete
66223a710990: Pull complete
db38d58ec8ab: Pull complete
Digest: sha256:99bac5bf83633e3c7399aed725c8415e7b569b54e03e4599e580fc9cdb7c21ab
Status: Downloaded newer image for openjdk:11
docker.io/library/openjdk:11
$ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 docker-compose run --rm   my-test-service  uname -a
Linux fa5509e3ab6f 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 x86_64 GNU/Linux

[JeancarloBarrios]

Do you know if there is a solution to this error?

[ndeloof]

I proposed an additional verification to avoid running the wrong platform image if another one exists for same tag on daemon local store. Please note this store can't have multiple platforms stored under same tag, so we can't automatically pull the desired platform, until containerd image store has been enabled - which is only available as an experimental feature in Docker Desktop, so not something we can use as default behavior yet