Bake: target override using file breaks references to its tags list from other target

[ReneNijhuis]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Error upon parsing bakefile and an override file about a targets tag list being null when referenced from another target. Occurs when the target override does not specify an overriding tag list but overrides different values.
The error occurs on the second target referencing the firsts tags but when baking only the first target its tag list is not empty.

The error only occurs when overriding using a file, using --set from the CLI correctly overrides the target without error.

Expected behaviour

The build works with the second target correctly using the referenced tag from the taglist. Using a file to override the first target should work the same as using the CLI --set flag.

Actual behaviour

An error is thrown upon parsing the bakefiles. Can be worked around by either specifying the override on the CLI or by copying the target.tags entry from the base file to the override file. When building only the first, overriden, target the tags list is not null or empty but when building the second target which references the first the error occurs.

Buildx version

github.com/docker/buildx v0.22.0-desktop.1 b45d7a53aa26a5e76aae051acdb15775c0e178da

Docker info

Client:
 Version:    28.0.4
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v1.1.3
    Path:     C:\Program Files\Docker\cli-plugins\docker-ai.exe
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.22.0-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  cloud: Docker Cloud (Docker Inc.)
    Version:  0.2.20
    Path:     C:\Program Files\Docker\cli-plugins\docker-cloud.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.34.0-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.38
    Path:     C:\Program Files\Docker\cli-plugins\docker-debug.exe
  desktop: Docker Desktop commands (Beta) (Docker Inc.)
    Version:  v0.1.6
    Path:     C:\Program Files\Docker\cli-plugins\docker-desktop.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scout: Docker Scout (Docker Inc.)
    Version:  v1.17.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 12
 Server Version: 28.0.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 753481ec61c7c8955a23d6ff7bc8e4daed455734
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 5.15.167.4-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.613GiB
 Name: docker-desktop
 ID: 4b409789-cfee-4f17-a1a6-76f557128e23
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=npipe://\\.\pipe\docker_cli
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Builders list

NAME/NODE           DRIVER/ENDPOINT     STATUS    BUILDKIT   PLATFORMS
default             docker
 \_ default          \_ default         running   v0.20.2    linux/amd64 (+4), linux/arm64, linux/arm (+2), linux/ppc64le, (3 more)
desktop-linux*      docker
 \_ desktop-linux    \_ desktop-linux   running   v0.20.2    linux/amd64 (+4), linux/arm64, linux/arm (+2), linux/ppc64le, (3 more)
test                                    error

Cannot load builder test: protocol not available

Configuration

The base docker-bake.hcl:

group "default" {
    targets = [ "tgt2" ]
}

target "tgt1" {
    dockerfile-inline = "FROM alpine:latest\nARG SOME_ARG\nENV IMG_VERSION=$${SOME_ARG:-1}"
    tags = ["main:latest", "primary-image:latest"]
}

target "tgt2" {
    dockerfile-inline = "FROM primary\nENV IMG_CONTINUED=true"
    contexts = { primary = "target:tgt1" }
    tags = [ target.tgt1.tags.0, "secondary-image:latest" ]
}

The override override.bake.hcl:

target "tgt1" {
    args = {
        SOME_ARG = 3
    } 
}

In the first command the override works correctly in the second it fails. Included in the log are the outputs of both since the output of the second is expected to be equal to the first.

docker buildx bake -f docker-bake.hcl --print  --set tgt1.args.SOME_ARG=3
docker buildx bake -f docker-bake.hcl -f override.bake.hcl --print

Build logs

$ ~/bake-test ❯ docker buildx bake -f docker-bake.hcl --print --set tgt1.args.SOME_ARG=3
[+] Building 0.0s (1/1) FINISHED
 => [internal] load local bake definitions                                              0.0s
 => => reading docker-bake.hcl 391B / 391B                                              0.0s
{
  "group": {
    "default": {
      "targets": [
        "tgt2"
      ]
    }
  },
  "target": {
    "tgt1": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM alpine:latest\nARG SOME_ARG\nENV IMG_VERSION=${SOME_ARG:-1}",
      "args": {
        "SOME_ARG": "3"
      },
      "tags": [
        "main:latest",
        "primary-image:latest"
      ]
    },
    "tgt2": {
      "context": ".",
      "contexts": {
        "primary": "target:tgt1"
      },
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM primary\nENV IMG_CONTINUED=true",
      "tags": [
        "main:latest",
        "secondary-image:latest"
      ]
    }
  }
}

$ ~/bake-test ❯ docker buildx bake -f docker-bake.hcl -f override.bake.hcl --print
[+] Building 0.0s (1/1) FINISHED
 => [internal] load local bake definitions                                              0.0s
 => => reading docker-bake.hcl 391B / 391B                                              0.0s
 => => reading override.bake.hcl 58B / 58B                                              0.0s
docker-bake.hcl:13
--------------------
  11 |         dockerfile-inline = "FROM primary\nENV IMG_CONTINUED=true"
  12 |         contexts = { primary = "target:tgt1" }
  13 | >>>     tags = [ target.tgt1.tags.0, "secondary-image:latest" ]
  14 |     }
--------------------
ERROR: docker-bake.hcl:13,30-32: Attempt to index null value; This value is null, so it does not have any indices., and 1 other diagnostic(s)

Additional info

Occurs not only on Docker Desktop under Windows but also on Ubuntu 22.04 with Buildx version github.com/docker/buildx 0.14.1 0.14.1-0ubuntu1~22.04.1. Error is the same on both and occurs before trying to connect to the Docker socket. Provided the info of the Windows install since the error still occurs on its newer version.

[tonistiigi]

It does work when you reverse the order of the files, but that could of course cause some other error if you change the content to a different one.

» docker buildx bake -f override.bake.hcl -f docker-bake.hcl  --print
[+] Building 0.0s (1/1) FINISHED
 => [internal] load local bake definitions                                                                                                                                                                                                    0.0s
 => => reading override.bake.hcl 59B / 59B                                                                                                                                                                                                    0.0s
 => => reading docker-bake.hcl 392B / 392B                                                                                                                                                                                                    0.0s
{
  "group": {
    "default": {
      "targets": [
        "tgt2"
      ]
    }
  },
  "target": {
    "tgt1": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM alpine:latest\nARG SOME_ARG\nENV IMG_VERSION=$${SOME_ARG:-1}",
      "args": {
        "SOME_ARG": "3"
      },
      "tags": [
        "main:latest",
        "primary-image:latest"
      ],
      "output": [
        {
          "type": "cacheonly"
        }
      ]
    },
    "tgt2": {
      "context": ".",
      "contexts": {
        "primary": "target:tgt1"
      },
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM primary\nENV IMG_CONTINUED=true",
      "tags": [
        "main:latest",
        "secondary-image:latest"
      ]
    }
  }
}

[ReneNijhuis]

Sure reversing the order works in my minimal example because the base doesn't define the SOME_ARG and they are simply merged. As soon as the base does define the overriden key, then the reversed order renders the override useless since the value from the base will be the last one defined.