Docker swarm does not recognize service ports specification from compose config

[jonaslb]

Description

Using a regular docker compose definition for a stack, I wanted to use .env file variables, which however is not supported with docker stack deploy .., so I wanted to use the workaround docker stack deploy -c <(docker compose configure) stackname.

This revealed however an incompatibility that I think is a missing implementation in Swarm: services.servicename.ports.0.published is generated as a string from docker compose config, most likely because it supports specifying a port range. However this means that docker stack deploy fails with the message: services.servicename.ports.0.published must be a integer.

Reproduce

Take the following minimal compose example:

services:
  my_service:
    image: nginx
    ports:
    - target: 80
      published: 9125

Try to execute docker compose config - you will see that it outputs "9125", a string, as the published port.
Now try to execute docker stack deploy -c <(docker compose config) mystack, and observe the error message.

Expected behavior

Swarm should interpret the published port number properly and deploy the stack.

docker version

$ docker version
Client: Docker Engine - Community
 Version:           23.0.0
 API version:       1.41 (downgraded from 1.42)
 Go version:        go1.19.5
 Git commit:        e92dd87
 Built:             Wed Feb  1 17:47:51 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 17:59:49 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.9
  GitCommit:        1c90a442489720eec95342e1789ee8a5e1b9536f
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.15.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 26
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: woua6888bdr2pa6qs2jwl7nnu
  Is Manager: true
  ClusterID: t5vjx0ip74vvgf58v3xg0pbks
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 172.19.158.143
  Manager Addresses:
   172.19.158.143:2377
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1c90a442489720eec95342e1789ee8a5e1b9536f
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.15.79.1-microsoft-standard-WSL2
 Operating System: Ubuntu 22.10
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.62GiB
 Name: LPCP2158
 ID: 7CGJ:WOZO:CHP4:FBWC:PIJ4:JX4C:DQU7:I27X:IHY2:HCZQ:2CH3:BHIB
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

[jonaslb]

I can see other differences too, such as service.servicename.depends_on needing to be a list, wheras docker configure seems to output a dictionary (with additional info on required dependency service state). Maybe this is a well known and more fundamental incompatibility between the docker compose configure output and expected input for docker stack deploy?

[neersighted]

I don't think this is necessarily a bug/a suggested use-case, as Swarm uses a different implementation of the compose spec in the CLI, and I think ultimate your issue comes from blending the compose spec with stack deploy's minimal implementation of 3.x.

Ultimately making stack deploy use compose-go, or replacing stack deploy with a Swarm-aware compose, are most likely what you would want to see here. But keep in mind that docker compose configure is not meant to have output usable by even other compose-compatible tools, it's simply a way to 'render' a spec.

[thaJeztah]

Yeah, I think this was somewhat of a breaking change in the compose-spec; some related discussions in

• mark explicitly published port as string to avoid user confusion docker/docs#15375
• published can either be a port(int) or range (string) compose-spec/compose-go#230
• v2.3.x config yaml output contains ports as strings docker/compose#9306