[BUG] !reset handling doesn't comply with specification

[EtienneMiret]

Description

According to the !reset reference:

custom YAML tag !reset can be set to override value set by the overriden Compose file, and replace with default value or null on target attribute.

(emphasis is mine)

However running docker compose -f compose-0.yaml -f compose-1.yaml up -d with the below files yields the error:

validating compose-1.yaml: services.web.volumes must be a list

compose-0.yaml:

services:

  web:
    image: nginx
    ports:
      - "8080:80"
    volumes:
      - webroot:/usr/share/nginx/html:ro

volumes:
  webroot:

compose-1.yaml:

services:

  web:
    volumes: !reset

volumes: !reset

According to #10589 (comment), we are supposed to add the default value after the !reset tag. For one thing, if instead of compose-1.yaml above, I use compose-2.yaml below, it still doesn’t work. I get no error, but the webroot volume is created. Furthermore, this is not what the spec states.

compose-2.yaml

services:

  web:
    volumes: !reset []

volumes: !reset {}

Now, to be fair, the spec says that !reset replaces either by the default value, either by null. I understand this as Replace by the default value if it exists, otherwise replace by null. But maybe replacing by the invalid value null when there is a default value available is allowed. It wouldn’t make sense IMHO.

Steps To Reproduce

1. Using the two yaml file compose-0.yaml and compose-1.yaml specified above.
2. Run docker compose -f compose-0.yaml -f compose-1.yaml up -d.
3. See error: validating compose-1.yaml: services.web.volumes must be a list

Compose Version

Docker Compose version v2.17.3

Docker Environment

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.4
    Path:     /Users/etienne/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.17.3
    Path:     /Users/etienne/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/etienne/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.19
    Path:     /Users/etienne/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.4
    Path:     /Users/etienne/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/etienne/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/etienne/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  v0.10.0
    Path:     /Users/etienne/.docker/cli-plugins/docker-scout

Server:
 Containers: 4
  Running: 0
  Paused: 0
  Stopped: 4
 Images: 62
 Server Version: 23.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc version: v1.1.5-0-gf19387a
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.773GiB
 Name: docker-desktop
 ID: 7ae811a5-43d5-42ad-bb0b-06becb3730c2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[ndeloof]

!reset is a yaml tag, still the value need to be set to the correct type, even those are ignored, to make a valid compose file:
So you indeed need to set:

services:
  web:
    volumes: !reset [] # empty array
volumes: !reset {} # nil mapping

with such a file, volume get removed as expected:

docker compose -f compose-0.yaml -f compose-2.yaml config --no-normalize
name: test
services:
  web:
    image: nginx
    networks:
      default: null
    ports:
    - mode: ingress
      target: 80
      published: "8080"
      protocol: tcp

I get no error, but the webroot volume is created.

Please use docker compose config to double check volume is actually removed from model

[EtienneMiret]

I get no error, but the webroot volume is created.

Please use docker compose config to double check volume is actually removed from model

You’re right here. When I saw the volume existed, it must have been there from a previous docker compose invocation that used a different config.

[EtienneMiret]

!reset is a yaml tag, still the value need to be set to the correct type, even those are ignored, to make a valid compose file

If the value needs to be set to some specific type, it should be documented. I’m reporting a discrepancy between specification and implementation. I believe in such case it’s better to fix the implementation rather than the specification, but after all HTML 5 and CSS 2 did it the other way, so…

[ndeloof]

I’m reporting a discrepancy between specification and implementation

actually the specification uses a single example with an implicit null value which makes it confusing. But there's no discrepancy here, just misunderstanding on use of yaml tags (which is something most of us never heard about :P)

I've opened a PR on the compose specification to better document use of this syntax