$PATH and ${PATH} should always mean the same thing within a Dockerfile ENV instruction

[papb]

Description

$PATH and ${PATH} should always mean the same thing within a Dockerfile ENV instruction.

Steps to reproduce the issue:

Using Mac (Catalina 10.15.5) with Docker v20.10.8:

1. Create a Dockerfile with ENV FOO=$PATH and ENV BAR=${PATH}
2. Build this Dockerfile
3. Run bash on it and check echo $FOO and echo $BAR

Describe the results you received:

Observe that both variables have different values. The former has the container PATH, while the latter has the host's PATH.

Describe the results you expected:

Both variables should have the container PATH.

Additional information you deem important (e.g. issue happens only occasionally):

This does not happen in Docker Desktop for Windows Docker version 19.03.13, build 4484c46d9d with MinGW from Git Bash 2.29.2.windows.2.

I am filing this issue according to this comment on stackoverflow. I do not have access to a Mac (Catalina 10.15.5).

[dankirkd]

As the writer of the comment I can confirm this issue.

[thaJeztah]

Hmm.. I don't see how they should be different, unless the Dockerfile has a ARG PATH, and a build-arg is passed).

Here's PATH on my host;

echo $PATH
/usr/local/opt/python@3.8/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/go/bin:/usr/local/opt/go/libexec/bin:/Users/sebastiaan/go/bin

Dockerfile:

FROM ubuntu
ENV FOO=$PATH
ENV BAR=${PATH}
RUN echo $FOO
RUN echo $BAR

Building with the classic (non-buildkit) builder;

DOCKER_BUILDKIT=0 docker build --no-cache --progress=plain -t issue-42863 .
...
Step 4/5 : RUN echo $FOO
 ---> Running in 32b290d790bf
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
...
Step 5/5 : RUN echo $BAR
 ---> Running in 91e3308aae27
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Inspecting the image to verify the expected values are set in the env-var (output formatted with jq for readability);

docker image inspect --format "{{ json .Config.Env }}" issue-42863
[
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "FOO=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "BAR=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
]

And when running the image:

docker run --rm issue-42863 bash -c 'echo $FOO; echo $BAR; echo "env:"; env'
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
env:
HOSTNAME=3524e0e81718
PWD=/
FOO=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOME=/root
BAR=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SHLVL=0
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/env

Trying the same with BuildKit enabled:

DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t issue-42863 .
...
#5 [2/3] RUN echo /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#5 sha256:ab3ea588503bb1a7adeb26327ca6bd9314848f25bc1e71603549203d0124cfa2
#5 0.260 /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
...
#6 [3/3] RUN echo /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#6 sha256:1e0a5ecab493edb1c4d795d7a165da7d5585d68a20ace07fbaf9caef20c0e527
#6 0.304 /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
...

Inspecting the image to verify the expected values are set in the env-var (output formatted with jq for readability);

docker image inspect --format "{{ json .Config.Env }}" issue-42863
[
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "FOO=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "BAR=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
]

And when running the image:

docker run --rm issue-42863 bash -c 'echo $FOO; echo $BAR; echo "env:"; env'
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
env:
HOSTNAME=bd538c962aeb
PWD=/
FOO=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOME=/root
BAR=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SHLVL=1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/env

[thaJeztah]

Do you have exact steps to reproduce? Does it reproduce with the steps I wrote above?

[thaJeztah]

closing as this one went stale

[dankirkd]

Do you have exact steps to reproduce? Does it reproduce with the steps I wrote above?

Yes. As provided in the description above.

[thaJeztah]

Yes. As provided in the description above.

Those "roughly" describe steps, but not exact steps (the devil may be in the details)

Does the issue reproduce with the steps I wrote above in #42863 (comment) ?

[dankirkd]

An interesting comment on this was posted at https://stackoverflow.com/questions/27093612/in-a-dockerfile-how-to-update-path-environment-variable/65119275?noredirect=1#comment131137480_65119275 :

The inconsistency you are facing isn't Docker it's Linux shell (BASH/SH) related. Those ${} are native to Linux scripting. Likely docker has simply integrated with the native shell of the OS so when you put this into the file it will be interpreted by the host/system OS depending on the one you use. If you want the details I suggest you look into bash scripting.

I wonder if that sheds any useful light on the matter.

[thaJeztah]

I wonder if that sheds any useful light on the matter.

ENV definitely doesn't use the host's shell (it's a metadata-only step in the Dockerfile). RUN steps evaluate environment variables using the shell that's defined for the image (default is a POSIX /bin/sh shell), but executed inside a container, so not using environment variables from the host, only environment variables that are defined in the Dockerfile (or the image).

@dankirkd I see the original reporter was using "MinGW from Git Bash 2.29.2.windows.2" - are you also running on Windows in a similar environment?

If so, then I suspect it's MinGW / MSYS2 that's causing the output to be modified. You can disable that behaviour by setting the MSYS_NO_PATHCONV=1 environment variable in your shell; see https://stackoverflow.com/questions/7250130/how-to-stop-mingw-and-msys-from-mangling-path-names-given-at-the-command-line and git-for-windows/git#577 (comment)

[dankirkd]

@dankirkd I see the original reporter was using "MinGW from Git Bash 2.29.2.windows.2" - are you also running on Windows in a similar environment?

@thaJeztah - No, I run on a Mac.

[papb]

@thaJeztah Hello! To clear up any confusion, I opened this issue "on behalf" of this comment on StackOverflow. I did not experience the issue myself. I was so astonished with the comment that I thought I should file the issue anyway.

I see the original reporter was using "MinGW from Git Bash 2.29.2.windows.2"

This does not happen in Docker Desktop for Windows Docker version 19.03.13, build 4484c46d9d with MinGW from Git Bash 2.29.2.windows.2.

@thaJeztah Apart from being astonished and opening the issue, I tried to reproduce it, but I didn't have a Mac, so I tried on Windows, and the issue did not occur. I decided to add this additional info just in case it helped debugging the problem. Sorry if it added more confusion instead 😅

Also, after reading all comments so far, I suggest reopening the issue :)

[papb]

@thaJeztah did you run this on a Mac? If so, which version?

[papb]

Do you have exact steps to reproduce? Does it reproduce with the steps I wrote above?

Yes. As provided in the description above.

@dankirkd I think the "yes" you said here ended up being ambiguous (because they asked two questions). To me it seems your "yes" refers only to the first question.

@dankirkd Can you please explicitly answer again if the issue happens for you when running precisely the steps from their comment?

[papb]

Also, @dankirkd, are you still using a Mac Catalina 10.15.5 or do you have a different version now?