docker-compose V2 fails for port range and replicas>1 where V1 would succeed

[manuel-koch]

Description

Using multiple replicas of a container where each replica should use one of the ports selected by a port-range:

version: "3.4"
services:
  dummy:
    image: busybox
    command: ["sh", "-c", "while true ; do sleep 3 ; date ; done"]
    deploy:
      replicas: 2
    ports: 
      - "6005-6015:5000"

When starting up these containers with docker-compose V1 the container instances use port-mapping where each container picks one host-port from the selected port range to map to internal container port.

$ docker-compose ps
            Name                          Command               State                    Ports
----------------------------------------------------------------------------------------------------------------
docker-compose-issue_dummy_1   sh -c while true ; do slee ...   Up      0.0.0.0:6007->5000/tcp,:::6007->5000/tcp
docker-compose-issue_dummy_2   sh -c while true ; do slee ...   Up      0.0.0.0:6008->5000/tcp,:::6008->5000/tcp

When starting up these containers with docker-compose V2 the containers use all ports from the selected port range to map to internal container port, resulting in error like the following for the second instance of that service:

Error response from daemon: driver failed programming external connectivity on endpoint docker-compose-issue_dummy_1 (6d9cff4937b3b6946e0bb949a36362cf91477f841be34ac9bd2693453b6e8837): Bind for 0.0.0.0:6009 failed: port is already allocated

Steps to reproduce the issue:

1. Switch to docker-compose V1, docker-compose disable-v2
2. Startup containers, docker-compose up -d
3. List running containers and the port mapping they use, docker-compose ps
4. Shutdown service, docker-compose down
5. Switch to docker-compose V2, docker-compose enable-v2
6. Startup containers, docker-compose up -d
7. docker-compose fails to startup because of port mapping clash

Describe the results you received:

docker-compose.yaml only seems to work for V1 and not V2.

Describe the results you expected:

I would expect that docker-compose.yaml can be run by V1 and V2 alike.

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.16.4
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:22 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:58 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  compose: Docker Compose (Docker Inc., 2.0.0-beta.3)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 30
  Running: 10
  Paused: 0
  Stopped: 20
 Images: 136
 Server Version: 20.10.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.25-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 3
 Total Memory: 11.7GiB
 Name: docker-desktop
 ID: 65BF:GFI6:S3ZY:URLI:EHO5:GFN5:PT7Y:Q7JW:ACRV:FJCE:YM5X:2YFM
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 159
  Goroutines: 157
  System Time: 2021-06-24T14:52:06.2642643Z
  EventsListeners: 5
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS ECS, Azure ACI, local, etc.):

local, MacOS BigSur 11.4

[manuel-koch]

When running the example from above the V1 docker-compose will issue a warning but proceed without an error:
WARNING: The "dummy" service specifies a port on the host. If multiple containers for this service are created on a single host, the port will clash.

Is it possible that V2 is more restricted and hence just fails instead of reporting a warning like V1 ?

Additionally the docker-compose ports documentation is not very clear about this apparently working port-range feature in V1, i.e. automatically assign a different port from the given range of ports to multi-replica-services.

[ndeloof]

V2 indeed doesn't reproduce this V1 warning, this is something we should consider.

documentation is not very clear about this apparently working port-range feature in V1
Indeed, I didn't know about it :D

Compose v1 creates containers sequentially, as an equivalent to docker run. Then the port specification is passed to engine. According to https://docs.docker.com/engine/reference/run/#expose-incoming-ports:
"When specifying a range for hostPort only, the
containerPort must not be a range. In this case the
container port is published somewhere within the
specified hostPort range. (e.g., -p 1234-1236:1234/tcp)".

Compose v2 creates containers in parallel, so this engine feature doesn't apply and this results in a port conflict. Seems we will have to drop this (premature?) optimization to restore this feature.

[infor7]

Hi

I have same problem on V2. The host ports are assigned to one container instead of distributing them among all replicas.

d0ac8e9625f3   bitnami/redis-sentinel:latest   "/opt/bitnami/script…"   About a minute ago   Up 53 seconds   0.0.0.0:26379->26379/tcp, :::26379->26379/tcp, 0.0.0.0:26380->26379/tcp, 0.0.0.0:26381->26379/tcp, :::26380->26379/tcp, :::26381->26379/tcp   os24_support_redis-sentinel_3

On V1 replicas works just fine:

1b5d5d7fbe67   bitnami/redis-sentinel:latest   "/opt/bitnami/script…"   7 seconds ago    Up 3 seconds    0.0.0.0:26379->26379/tcp, :::26379->26379/tcp                                  os24_support_redis-sentinel_2
4fd28e9d6e21   bitnami/redis-sentinel:latest   "/opt/bitnami/script…"   7 seconds ago    Up 3 seconds    0.0.0.0:26381->26379/tcp, :::26381->26379/tcp                                  os24_support_redis-sentinel_3
2c0a7016ba04   bitnami/redis-sentinel:latest   "/opt/bitnami/script…"   7 seconds ago    Up 3 seconds    0.0.0.0:26380->26379/tcp, :::26380->26379/tcp                                  os24_support_redis-sentinel_1

[seonglae]

This is still problem

Linux dc165-test 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Docker version 20.10.9, build c2ea9bc
Docker Compose version v2.0.1

[ndeloof]

ran a quick test, and this actually isn't an issue with replicas. Even using replicas: 1 container is bound to the whole range, not "somrwhere within the specified range`

0.0.0.0:6005->5000/tcp, 0.0.0.0:6006->5000/tcp, 0.0.0.0:6007->5000/tcp, 0.0.0.0:6008->5000/tcp, 0.0.0.0:6009->5000/tcp, 0.0.0.0:6010->5000/tcp, 0.0.0.0:6011->5000/tcp, 0.0.0.0:6012->5000/tcp, 0.0.0.0:6013->5000/tcp, 0.0.0.0:6014->5000/tcp, 0.0.0.0:6015->5000/tcp

.. so there's something wrong with compose v2 passing port specification to the engine.

[ndeloof]

confirmed.
compose-v1 creates container as:

"PortBindings": {
      "5000/tcp": [
        {
          "HostIp": "",
          "HostPort": "6005-6015"
        }
      ]
    },

compose-v2 creates container as:

"PortBindings": {
      "5000/tcp": [
        {
          "HostIp": "",
          "HostPort": "6005"
        },
        {
          "HostIp": "",
          "HostPort": "6006"
        },
        {
          "HostIp": "",
          "HostPort": "6007"
        },
...

[ndeloof]

the issue is in compose-go, which "extends" 6005-6015:5000 into a list of bindings for each and every values in the range

[C-Powers]

I'm not sure if this is the same problem, but researching how to solve my own problem took me here. I was having a similar problem on a Mac,

christa % docker compose up
Recreating recorder ... error
ERROR: for recorder  Cannot start service recorder: Ports are not available: listen tcp 0.0.0.0:5000: bind: address already in use

and I was able to fix it by checking:

christa % sudo lsof -i -P -n | grep 5000                                      
ControlCe 56829         christa   22u  IPv4 0xce0b919697e39c8f      0t0    TCP *:5000 (LISTEN)
ControlCe 56829         christa   23u  IPv6 0xce0b919684ffd2b7      0t0    TCP *:5000 (LISTEN)

and then turning off the Airplay Receiver Service in System Preferences -> Sharing

[ndeloof]

compose-go both extends port range to a list of ports, but also assumes this is expected
and types are designed to only accept an actual port (uint32) not range.

I guess we are stuck here :'(

[ndeloof]

This will also require some clarification on https://github.com/compose-spec/compose-spec/blob/master/spec.md#ports if we want this to be addressed

• the compose spec schema does not allow use of a string as published port value (https://github.com/compose-spec/compose-spec/blob/master/schema/compose-spec.json#L321)

[Hazmi35]

Sorry for the bump, any updates?

[rs-renato]

the same issue here, I force myself to downgrade to v1