[BUG] 'compose' is not a docker command.

[ryanlambert-wk]

Description

After fresh install of Docker Desktop (either via .dmg or homebrew cask) attempting to use docker-compose or docker compose results in the following error.

% docker-compose
docker: 'compose' is not a docker command.
See 'docker --help'

% docker compose
docker: 'compose' is not a docker command.
See 'docker --help'

It seems that the application installs to the /Applications/Docker.app/Contents/Resources/bin/ directory then creates symlinks to the appropriate binaries in /usr/local/bin (seen below).

lrwxr-xr-x@  1 root  wheel      54 Jul  2 17:14 docker -> /Applications/Docker.app/Contents/Resources/bin/docker
lrwxr-xr-x@  1 root  wheel      62 Jul  2 18:32 docker-compose -> /Applications/Docker.app/Contents/Resources/bin/docker-compose
lrwxr-xr-x@  1 root  wheel      80 Jul  2 17:14 docker-compose-v1 -> /Applications/Docker.app/Contents/Resources/bin/docker-compose-v1/docker-compose

The docker and docker-compose-v1 commands seem to work fine as seen below:

 % which docker-compose-v1
/usr/local/bin//docker-compose-v1

% /Applications/Docker.app/Contents/Resources/bin/docker-compose-v1/docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

 % docker-compose-v1
Define and run multi-container applications with Docker.

Usage:
  docker-compose [-f <arg>...] [--profile <name>...] [options] [--] [COMMAND] [ARGS...]
  docker-compose -h|--help

Options:
  -f, --file FILE             Specify an alternate compose file
                              (default: docker-compose.yml)
  -p, --project-name NAME     Specify an alternate project name
                              (default: directory name)
  --profile NAME              Specify a profile to enable
  -c, --context NAME          Specify a context name
        ... truncated ...

However, even when going directly to the source binary, docker-compose doesn't seem to work. which shows the correct location of the symlink to the binary, but the binary itself seems non-functional. Instead, the functional v2 binary seems to now exist in ~/.docker/cli-plugins/ by default.

% which docker-compose
/usr/local/bin//docker-compose

% /Applications/Docker.app/Contents/Resources/bin/docker-compose
docker: 'compose' is not a docker command.
See 'docker --help'

% ~/.docker/cli-plugins/docker-compose version
Docker Compose version v2.19.0

% ~/.docker/cli-plugins/docker-compose        

Usage:  docker compose [OPTIONS] COMMAND

Define and run multi-container applications with Docker.
     ... truncated ...

I can fix this issue by running the following commands to replace the broken symlink.

sudo rm /usr/local/bin/docker-compose
sudo ln -s ~/.docker/cli-plugins/docker-compose /usr/local/bin/docker-compose

While this works, it unfortunately causes docker to complain about modified configurations, and if at any point you select Re-apply configurations it will reset the symlink to point to the non-functioning binary.

Steps To Reproduce

1. Install the apple silicon version of docker desktop on an M2 Max MacBook Pro running macOS Ventura 13.4.1.
2. Attempt to use the docker-compose command.

Compose Version

% docker-compose version
docker: 'compose' is not a docker command.
See 'docker --help'

% ~/.docker/cli-plugins/docker-compose version
Docker Compose version v2.19.0

Docker Environment

% docker info
Client:
 Version:    24.0.2
 Context:    default
 Debug Mode: false
 Plugins:
WARNING: Plugin "/Users/ryanlambert/.wk/config/docker/cli-plugins/docker-compose" is not valid: failed to fetch metadata: fork/exec /Users/ryanlambert/.wk/config/docker/cli-plugins/docker-compose: exec format error

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 8
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.667GiB
 Name: docker-desktop
 ID: 5346f842-3a33-43f2-a6e2-b4b7dde691be
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

OS Version: macOS Ventura 13.4.1

[ndeloof]

Can you please check you have feature file set to enable compose v2 ?

$ cat ~/.docker/features.json 
{
  "composeV2": "enabled"
}

[ryanlambert-wk]

Can you please check you have feature file set to enable compose v2 ?

$ cat ~/.docker/features.json 
{
  "composeV2": "enabled"
}

Confirmed that it is enabled.

[ryanlambert-wk]

Just a heads up, I installed the latest version of docker which included a newer version of docker compose and this problem doesn't seem to exist anymore since docker-compose worked right out of the box.

% docker info
Client:
 Version:    24.0.2
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.0
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.19.1
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.16.1
    Path:     /Users/ryanlambert/.docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.667GiB
 Name: docker-desktop
 ID: 4a04ddef-4bc2-47aa-8c29-d379a8e4b9ad
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

[ryanlambert-wk]

Hmm, not sure what happened but somehow it's broken again.

% docker info
Client:
 Version:    24.0.2
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.667GiB
 Name: docker-desktop
 ID: 4a04ddef-4bc2-47aa-8c29-d379a8e4b9ad
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

[milas]

Moved this to the Docker Desktop for macOS issue tracker, as this looks like it's specific to that versus a generic Compose issue.

Can you upload & share a diagnostic ID?

https://docs.docker.com/desktop/troubleshoot/overview/

[ryanlambert-wk]

@milas The diagnostic ID is C118ACC3-6993-4208-BB75-CE7D114E09F1/20230808144017

[milas]

It looks like something is going on with your PATH. The diagnostic was unable to find the docker CLI, for example.

However, all the plugins & binaries are in the correct place according to your diag.

In the output you posted before:

WARNING: Plugin "/Users/ryanlambert/.wk/config/docker/cli-plugins/docker-compose" is not valid: failed to fetch metadata: fork/exec /Users/ryanlambert/.wk/config/docker/cli-plugins/docker-compose: exec format error

~/.wk/config/docker/cli-plugins is not the default path, are you using a custom DOCKER_CONFIG and installing things manually? That looks like you ended up with an amd64 binary there.

[ryanlambert-wk]

I installed docker using the "Apple Chip" binary from https://www.docker.com/products/docker-desktop/. I agree that it looks like everything is installed in the right places, but something isn't pointing to the ~/.wk/config/docker/cli-plugins directory properly, hence why the following commands work as a temporary fix.

sudo rm /usr/local/bin/docker-compose
sudo ln -s ~/.docker/cli-plugins/docker-compose /usr/local/bin/docker-compose

Maybe macOS is overwriting something? It seemed like in some cases the docker-compose command would work immediately after installation, but when I'd come back to my computer later, or possibly after a restart, it would no longer work.

[aheckman]

I encountered this as well. Sometimes docker-compose would work, sometimes not. Mostly not. My workaround is
export PATH="$HOME/.docker/cli-plugins;$PATH"
All the docker commands in $HOME/.docker/cli-plugins link to /Applications/Docker.app/Contents/Resources/cli-plugins
All the docker commands in /usr/local/bin link to /Applications/Docker.app/Contents/Resources/bin
HOWEVER ONLY "docker-compose" is in both of theses directories.
bin/docker-compose is 7043376 bytes (this is the "BAD" one)
cli-plugins/docker-compose is 58502608 bytes (this is the "GOOD" one)
I am running a fresh install of Docker Desktop 4.24.0 (122432) on Mac with Apple Silicon