special characters including hyphen is not working as identifier in docker build arg key

[vidhi-5794]

Description

special characters like hyphen, space is not working in docker build arg key (only under score _ works in key)

Reproduce

--build-arg 'key2-temp'='test value2' --build-arg 'key1 test'='test 1' is not working

ARG key2-temp
RUN echo $key2-temp # it prints -temp2 instead of its value.

Expected behavior

it should print test value2

docker version

Client:
 Cloud integration: v1.0.29
 Version:           20.10.21
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        baeda1f
 Built:             Tue Oct 25 18:01:18 2022
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.15.0 (93002)
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 17:59:41 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.10
  GitCommit:        770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.13.0)
  dev: Docker Dev Environments (Docker Inc., v0.0.5)
  extension: Manages Docker extensions (Docker Inc., v0.2.16)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.22.0)

Server:
 Containers: 16
  Running: 0
  Paused: 0
  Stopped: 16
 Images: 32
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 15.37GiB
 Name: docker-desktop
 ID: M6PA:C2UC:TUCT:7U7U:CXTF:TJN7:TSJO:PIET:JP4B:FXXD:6KIB:IY4Y
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

Is there any documentation which suggests that we cant use special characters including hyphen will not work in docker build arg key?

[thaJeztah]

Build-args are set as environment-variables in your RUN instructions, and (also see the ENV and ARG instructions in the Dockerfile).

When interpolating them it depends on the shell what's accepted, and how these are handled. The daemon (and OCI runtime) generally accept anything, except NUL characters; here's your earlier example, but using env to print the environment variables

docker build --progress=plain --build-arg 'key2-temp=hello world' -<<'EOF'
FROM alpine
ARG key2-temp
RUN env
EOF

Which shows this output for the env command;

#5 [2/2] RUN env
#5 0.150 key2-temp=hello world
#5 0.150 SHLVL=1
#5 0.150 HOME=/root
#5 0.150 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#5 0.150 PWD=/
#5 DONE 0.2s

Or, using docker run:

docker run --name foo --env "hello-world=hello-value" --env "hello world=hello value" --env "hello.world=hello value" alpine env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=1c5f844cc8c1
hello world=hello value
hello.world=hello value
hello-world=hello-value
HOME=/root

Or inspecting the container:

docker inspect --format '{{json .Config.Env}}' foo | jq .
[
  "hello world=hello value",
  "hello.world=hello value",
  "hello-world=hello-value",
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
]

The POSIX specification allows for most characters (except NUL) to be used, but recommends only using characters from the Portable Character Set; see http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html

Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation; applications shall tolerate the presence of such names. Uppercase and lowercase letters shall retain their unique identities and shall not be folded together. The name space of environment variable names containing lowercase letters is reserved for applications.

Unfortunately that also means that we cannot validate characters; we once did, but this caused issues as some tools depend on "invalid" env-vars (moby/moby#16585), so we had to revert (moby/moby#16608).

As my earlier example shows; these vars can be set in the environment, however, it depends on the shell how they are interpreted. In your $key2-temp example, you're using a hyphen; shells generally don't allow for a hyphen as part of an env-var, so they consider $key2 to be an environment variable (which is not set), and -temp to be a string following it.

Hyphens also have special meaning in most (POSIX compliant) shells; https://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html#tag_02_06_02

So for example ${key2-temp} is treated as;

• if $key2 is set, print its value
• if $key2 is set, but null, then don't print anything
• if $key2 is not set, then print "word" (temp)

docker run --rm --env "hello-world=hello-value" alpine sh -c 'echo ${hello-world}'
world

So in general, I would recommend sticking to a-z A-Z 0-9 and underscores (_) for variable names, as those would be most portable.

[vvoland]

Maybe the docs could be improved here?

\cc @dvdksn