[BUG] misleading status for non-host-mode containers

[Paraphraser]

Description

I've noticed something odd but I don't know whether it's actually a bug or the intended behaviour.

Consider this diff:

$ diff docker-compose.override.yml.save docker-compose.override.yml
4a5,14
>   default:
>     ipam:
>       config:
>         - subnet: 172.30.0.0/22
> 
>   nextcloud:
>     ipam:
>       config:
>         - subnet: 172.30.4.0/22
> 

In other words, docker-compose.override.yml defines specific subnets for two internal networks whereas the .save file does not.

The situation with those subnet definitions in force:

$ docker ps
CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS                   PORTS                    NAMES
a3d379eb6ba7   koenkk/zigbee2mqtt:latest   "docker-entrypoint.s…"   2 minutes ago   Up 2 minutes             0.0.0.0:8080->8080/tcp   zigbee2mqtt
c4e47e0f236d   iotstack-nodered            "./entrypoint.sh"        2 minutes ago   Up 2 minutes (healthy)   0.0.0.0:1880->1880/tcp   nodered
c484a9c1b5e9   grafana/grafana             "/run.sh"                2 minutes ago   Up 2 minutes (healthy)   0.0.0.0:3000->3000/tcp   grafana
ff51c96674a3   esphome/esphome             "/entrypoint.sh dash…"   2 minutes ago   Up 2 minutes (healthy)                            esphome
f127b0a10b85   iotstack-mosquitto          "/docker-entrypoint.…"   2 minutes ago   Up 2 minutes (healthy)   0.0.0.0:1883->1883/tcp   mosquitto
d8d58efc1c9d   influxdb:1.12               "/entrypoint.sh infl…"   2 minutes ago   Up 2 minutes (healthy)   0.0.0.0:8086->8086/tcp   influxdb

Now let's change the override file to be the one without the subnet definitions:

$ mv docker-compose.override.yml.save docker-compose.override.yml

Let's apply that:

$ docker compose up -d
WARN[0000] No services to build                         
[+] up 7/7
 ✔ Container nodered        Stopped 0.7s 
 ✔ Container zigbee2mqtt    Stopped 9.6s 
 ✔ Container grafana        Stopped 0.7s 
 ✔ Container influxdb       Stopped 1.0s 
 ✔ Container mosquitto      Stopped 0.4s 
 ✔ Network iotstack_default Created 0.1s 
 ✔ Container esphome        Running 0.0s

Compose says it has stopped five containers. The true situation:

$ docker ps
CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS                             PORTS                    NAMES
a3d379eb6ba7   koenkk/zigbee2mqtt:latest   "docker-entrypoint.s…"   3 minutes ago   Up 12 seconds                      0.0.0.0:8080->8080/tcp   zigbee2mqtt
c4e47e0f236d   iotstack-nodered            "./entrypoint.sh"        3 minutes ago   Up 12 seconds (health: starting)   0.0.0.0:1880->1880/tcp   nodered
c484a9c1b5e9   grafana/grafana             "/run.sh"                3 minutes ago   Up 12 seconds (healthy)            0.0.0.0:3000->3000/tcp   grafana
ff51c96674a3   esphome/esphome             "/entrypoint.sh dash…"   3 minutes ago   Up 3 minutes (healthy)                                      esphome
f127b0a10b85   iotstack-mosquitto          "/docker-entrypoint.…"   3 minutes ago   Up 12 seconds (healthy)            0.0.0.0:1883->1883/tcp   mosquitto
d8d58efc1c9d   influxdb:1.12               "/entrypoint.sh infl…"   3 minutes ago   Up 12 seconds (health: starting)   0.0.0.0:8086->8086/tcp   influxdb

To summarise:

1. We started with 6 containers running (esphome, grafana, influxdb, mosquitto, nodered and zigbee2mqtt).
2. We finished with the same six containers running.
3. After changing the override file, all bar esphome are reported as having been stopped. esphome being the exception is expected because that container runs in host mode so it isn't affected by internal network changes.
4. What's odd is that docker compose hasn't reported starting the stopped containers again, yet they're all clearly running.

Compare and contrast what happens if I pick a container at random (eg grafana) and add some dummy environment variable to its service definition. On the subsequent "up", I'll get:

 ✔ Container grafana     Recreated 0.5s 

In short, the expected behaviour is that all five non-host-mode containers would be reported as either "Recreated" or "Stopped" then "Created", rather than just "Stopped" which isn't actually true.

For the record, this pattern is the same in each direction. Going from "no subnets defined" to "subnets defined" has the same behaviour. It also doesn't matter whether the subnet definition occurs in the override file or the compose file. I just happen to be using the override file.

I hope that makes sense.

Steps To Reproduce

Implied in the above. In a compose file with a networks: section plus at least one non-host-mode container:

• either:

  1. Compose or override file without defined subnets.
  2. Down stack
  3. Up stack.
  4. Add subnet definitions to compose or override file.
  5. Up the stack.

• or:

  1. Compose or override file with defined subnets.
  2. Down stack
  3. Up stack.
  4. Remove subnet definitions from compose or override file.
  5. Up the stack.

Compose Version

$ docker compose version
Docker Compose version v5.0.1

$ docker-compose version
Docker Compose version v5.0.1

Docker Environment

Client: Docker Engine - Community
 Version:    29.1.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.30.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.0.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 6
  Running: 6
  Paused: 0
  Stopped: 0
 Images: 7
 Server Version: 29.1.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: local
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: dea7da592f5d1d2b7755e3a161be07f43fad8f75
 runc version: v1.3.4-0-gd6d73eb8
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.21-v8+
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.704GiB
 Name: sec-dev
 ID: f6406dbb-2fc1-4834-9606-5e826871aec6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
  ::1/128
 Live Restore Enabled: false
 Firewall Backend: iptables

Anything else?

No response

[Paraphraser]

I'll add that I'm aware that the nextcloud network is not in use by any container in this stack. The compose and override files are set up to support NextCloud if I want to run it.

[ndeloof]

I guess this is a UI issue: container need to be stopped so network configuration is updated (it is set by engine as container is started). To attach to containers and show startup logs, compose has to stop the UI before container actually get started, so you won't get a Started event reported. This is confusing, but as ps demonstrates, compose up actually put resources in the desired state.

This is a know issue with the UI, we hardly can fix as we can't mix the progress UI with container logs (until we fully redesign the TUI)