docker compose down does not remove all containers

[williamjacksn]

Description

Beginning with version 2.10.0, docker compose down does not remove containers that were created with docker compose run.

Steps to reproduce the issue:

# compose.yaml

services:
  hello:
    image: hello-world

» docker compose run hello
[+] Running 1/0
 ⠿ Network project_default  Created                                                                                   0.0s

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

» docker compose down
[+] Running 1/0
 ⠿ Network project_default  Removed                                                                                   0.1s

» docker container ls -a
CONTAINER ID   IMAGE         COMMAND    CREATED         STATUS                     PORTS     NAMES
e9b1f3b3fdc6   hello-world   "/hello"   7 seconds ago   Exited (0) 5 seconds ago             project_hello_run_ae1b89bc0eb9

Describe the results you received:

In version 2.10.0 and newer, docker compose down does not remove containers that were created with docker compose run.

Describe the results you expected:

In version 2.9.0, docker compose down will remove containers that were created with docker compose run.

Output of docker compose version:

Docker Compose version v2.11.2

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., v2.11.2)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: 1e7bb5b773162b57333d57f612fd72e3f8612d94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.14.287-215.504.amzn2.x86_64
 Operating System: Amazon Linux 2
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.37GiB
 Name: machine
 ID: MQRK:EG24:A2PO:TDV4:M5E2:H433:IEFW:LPJ4:PDMS:QHWC:OPSN:BEVO
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

[nicksieger]

Looks like this was introduced in f6e96dd, specifically the change to pkg/compose/down.go. I'm looking into why the change was made as it doesn't appear to be part of the rest of the changes in that commit or PR.

[atmecs123]

Facing some issue while "docker-compose down" with podman where container is still up and running even if executed down command to stop the container and it errors out with the below message in journalctl logs for the container service.
Sep 28 09:11:23 vflexa5340-10n2 svc-control.sh[312718]: settings is up-to-date
Sep 28 09:12:23 vflexa5340-10n2 svc-control.sh[312718]: An HTTP request took too long to complete. Retry with --verbose to obtain debug>
Sep 28 09:12:23 vflexa5340-10n2 svc-control.sh[312718]: If you encounter this issue regularly because of slow network conditions, consi>
Sep 28 09:12:23 vflexa5340-10n2 svc-control.sh[312718]: Attaching to settings
Sep 28 09:12:23 vflexa5340-10n2 systemd[1]: settings.service: Main process exited, code=exited, status=1/FAILURE
Sep 28 09:12:23 vflexa5340-10n2 systemd[1]: settings.service: Failed with result 'exit-code'.

Any idea what the issue is?

[nicksieger]

Ok, so it looks like compose v2 has required the --remove-orphans option to remove one-off containers like those created with docker compose run, but it previously only did that when supplying the -p/--project-name option. #9375 just fixed things and made it work the same whether the project name was specified or not.

At this point I'm not entirely certain what the specified behavior should be. V1 appears to remove containers without --remove-orphans, so I can understand how it might feel like this is a bug.

@williamjacksn can you start using --remove-orphans to achieve the behavior you want?

[williamjacksn]

@williamjacksn can you start using --remove-orphans to achieve the behavior you want?

It's one more thing to remember, but if this is how it is supposed to work then I guess I have to do it.

If this is the desired behavior, then the documentation is misleading:

Options:
     --remove-orphans    Remove containers for services not defined in the Compose file.

docker compose run <service-name> creates a container for a service that is defined in the Compose file.

[nicksieger]

For now that's your workaround. I'll see if we can figure out what the intended behavior should be and either update the --remove-orphans documentation or change the behavior.

[ndeloof]

"orphans" containers is compose terminology for containers which declare a service that is not part of the current compose model, typically because user edited his compose file in the meantime.
"one-off" containers created by run command never are considered related to a service, as those are designed to run single tasks, not to be used as long-running services. So any one-off container, labelled for the current compose project, is considered "orphaned" as it can't be linked to a selected service.

terminology is far from being perfect, but this is our legacy :)

[dbaltor]

Facing the same issue of a container not being stopped by docker-compose down even though I have created it via a compose file.

[glours]

@dbaltor was this container started with a docker compose run command? If so this is a one-off container which is not part of your current compose stack (see @ndeloof comment above).
Try docker compose down --remove-orphans to remove it

[dbaltor]

Hi @glours thanks for reaching out. Appreciate it.
Never mind, I figure it out. I'm specifying a compose file with a different name via docker-compose -f <some-name>.yml up but forgot to mention the file when using docker-compose down 🤦‍♂️

[Kjuly]

Hi @glours thanks for reaching out. Appreciate it. Never mind, I figure it out. I'm specifying a compose file with a different name via docker-compose -f <some-name>.yml up but forgot to mention the file when using docker-compose down 🤦‍♂️

Me too. That's why adding --remove-orphans also works, which "removes containers for services not defined in the Compose file". Because we're using different compose files..

So we need to use

$ docker compose -f another-compose.yml down