Document what are valid characters for a variable in a Dockerfile

[rcjsuen]

I was routed here from docker/docs#8835...

From cli/docs/reference/builder.md:

Environment variables are notated in the Dockerfile either with $variable_name or ${variable_name}. They are treated equivalently and the brace syntax is typically used to address issues with variable names with no whitespace, like ${foo}_bar.

From my testing, it seems like a variable name includes alphanumeric characters and the underscore...? If this is correct, could this please be mentioned in the Dockerfile reference?

[thaJeztah]

Thanks for opening this ticket; yes, perhaps some extra information could be added to describe what's valid.

It's slightly complicated to give a "single" answer, because it depends on whether or not the environment variable (or ARG) is used for substitution inside the Dockerfile, and where the substitution happens.

The Dockerfile parser ("front-end") itself handles substitution in various places, such as ENV (ENV Foo=$BAR), LABEL, FROM (see Environment replacement), but does not perform variable substitution itself in RUN commands (more on that below). For substitution handled by the Dockerfile parser, it (mostly) follows the POSIX specification, which defines valid characters for environment variables as follows;

Environment variable names used by the utilities in the Shell and Utilities volume
of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_'
(underscore) from the characters defined in Portable Character Set
and do not begin with a digit.

However for RUN instructions, variable substitution is not done by Docker itself; for those, Docker only sets the environment variable, and then starts a container to execute the RUN command. In that case, we try to be agnostic as possible and don't perform validation on the variable name. The reason for that is that, while the POSIX spec defines valid characters for environment variables (as described above), it also describes that implementations may differ from the above;

Other characters may be permitted by an implementation;
applications shall tolerate the presence of such names. Uppercase and lowercase
letters shall retain their unique identities and shall not be folded together.
The name space of environment variable names containing lowercase letters is
reserved for applications. Applications can define any environment variables
with names from this name space without modifying the behavior of the standard
utilities.

So, what's a valid environment-variable name for a RUN instruction fully depends on the image, and what's running inside the container. For these cases, Docker itself doesn't validate (other than trimming leading/trailing whitespace), and leaves it to the operating system to accept (or reject) setting it.

Here's a Dockerfile with some comments describing the above;

ARG FOO=world

# Here, Docker performs substitution
FROM myimage:${FOO-latest}
ARG FOO
ENV HELLO=${FOO}

# But in RUN instructions, substitution is handled by the shell used for the RUN
# command. The default shell for Linux images is "/bin/sh" and "cmd.exe" for
# Windows images, but can be customized in the image through the "SHELL"
# Dockerfile instruction.
RUN echo hello ${FOO}

# In the RUN below, substitution is done by PowerShell
RUN pwsh -c 'echo "hello $Env:FOO"'

# This also means that variable-names that may be invalid for substitution in
# the Dockerfile itself, may be acceptable for the RUN instruction. The example
# below sets an environment variable named "foo,bar":
ENV foo,bar=hello, I am foo,bar
RUN env | grep hello

Here's an example of some weird environment variable names;

docker build --progress=plain --no-cache -<<'EOF'
FROM alpine
ENV foo,bar="hello, I am foo,bar"
ENV hell#@0="hello world"
RUN env | grep hello
EOF

...
docker/cli#5 [2/2] RUN env | grep hello
docker/cli#5 0.509 foo,bar=hello, I am foo,bar
docker/cli#5 0.271 hell#@0=hello world
docker/cli#5 DONE 1.4s

FWIW, the same applies when running a container, for example;

docker run --rm -e "foo,bar=hello" alpine env | grep hello
foo,bar=hello

[thaJeztah]

Let me transfer this ticket to the https://github.com/docker/buildx repository, as we recently moved all build-related documentation to the BuildKit repository (and buildx is more closely related to building than this repository).