docker-compose stopped remotely updating the running containers with up --build command

[smartFunX]

Description of the issue

Starting from MacOS Docker Engine 20.10.3, docker-compose stopped remotely updating the running containers while running command:
docker-compose -H "ssh://..." up -d --build

Instead it builds and saves the image locally.

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.29.1, build c34c88b2
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Output of docker version

Client:
 Cloud integration: 1.0.14
 Version:           20.10.6
 API version:       1.41
 Go version:        go1.16.3
 Git commit:        370c289
 Built:             Fri Apr  9 22:46:57 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.6
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8728dd2
  Built:            Fri Apr  9 22:44:56 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker-compose config
(Make sure to add the relevant -f and other flags)

networks:
  docker-network:
    external: true
    name: docker-network
services:
  vis-app:
    build:
      context: /Users/user/data_visualizer
    environment:
      NODE_ENV: production
    extra_hosts:
    - host.docker.internal:host-gateway
    networks:
      docker-network:
        aliases:
        - vis-app
    ports:
    - published: 8084
      target: 8084
    restart: always
version: '3.9'

Steps to reproduce the issue

1. Run: docker-compose -H "ssh://..." up -d --build

Observed result

Docker compose just builds an image, but saves it locally and doesn't bring it up on the remote server.

Expected result

Docker should restart the remote containers with the latest built image.

Stacktrace / full error message

[+] Building 193.9s (15/15) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                       0.0s
 => => transferring dockerfile: 40B                                                                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                                                                          0.0s
 => => transferring context: 34B                                                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/node:12                                                                                                                                                 1.2s
 => [frontend 1/5] FROM docker.io/library/node:12@sha256:0d6b3522991d04ead0b119feaffafa6a8cd1b374218e9f997a1d4753b7064882                                                                                  0.0s
 => [internal] load build context                                                                                                                                                                          0.1s
 => => transferring context: 56.66kB                                                                                                                                                                       0.1s
 => CACHED [frontend 2/5] WORKDIR /tmp/react-app/                                                                                                                                                          0.0s
 => [frontend 3/5] COPY . .                                                                                                                                                                                0.2s
 => CACHED [stage-1 2/6] WORKDIR /usr/src/app                                                                                                                                                              0.0s
 => CACHED [stage-1 3/6] COPY server/package*.json ./                                                                                                                                                      0.0s
 => CACHED [stage-1 4/6] RUN npm install                                                                                                                                                                   0.0s
 => [stage-1 5/6] COPY server/ .                                                                                                                                                                           0.1s
 => [frontend 4/5] RUN npm install                                                                                                                                                                        78.8s
 => [frontend 5/5] RUN npm run build                                                                                                                                                                     108.1s
 => [stage-1 6/6] COPY --from=frontend /tmp/react-app/build ./build/                                                                                                                                       0.1s
 => exporting to image                                                                                                                                                                                     0.1s
 => => exporting layers                                                                                                                                                                                    0.1s
 => => writing image sha256:4af20e91adb333278e869522da4461d131ce6363d7d2183bd376bfeeacc8aa70                                                                                                               0.0s
 => => naming to docker.io/library/data_visualizer_vis-app                                                                                                                                           0.0s
Successfully built 4af20e91adb333278e869522da4461d131ce6363d7d2183bd376bfeeacc8aa70
data_visualizer_vis-app_1 is up-to-date

Additional information

MacOS Docker Engine version 20.10.3 and higher is affected.
Works well on 20.10.2 and lower.

[bikol]

This seams to impact more commands/options while working with remote DOCKER_HOST including variable in .env file, --context and -H flags.

Commands impacted:

• docker-compose build (while DOCKER_HOST variable is set in .env file)
• docker-compose up --build (while DOCKER_HOST variable is set in .env file)
• docker-compose -H "ssh://..." build
• docker-compose -H "ssh://..." up -d --build
• docker-compose --context "remote" build (remote is registered in docker context as "ssh://...")
• docker-compose --context "remote" up --build (remote is registered in docker context as "ssh://...")

As a workaround it is possible to define DOCKER_HOST in shell and then it works correctly

• DOCKER_HOST="ssh://..." docker-compose build
• DOCKER_HOST="ssh://..." docker-compose up --build

Or you can switch context manually using:

docker context use remote

Output of docker-compose version

docker-compose version 1.29.2, build unknown
docker-py version: 5.0.0
CPython version: 3.9.5
OpenSSL version: OpenSSL 1.1.1k  25 Mar 2021

Output of docker version

Client:
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.16.4
 Git commit:        f0df35096d
 Built:             Fri Jun  4 08:14:39 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.4
  Git commit:       b0f5bc36fe
  Built:            Fri Jun  4 08:14:24 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.5.2
  GitCommit:        36cc874494a56a253cd181a1a685b44b58a2e34a.m
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[smartFunX]

The solution with DOCKER_HOST doesn't work for me, I'm getting the same error as reported in this issue:
#8218

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it had not recent activity during the stale period.