Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENV command set env var not accepted by CMD / ENTRYPOINT (exec format) #4783

Closed
djsly opened this Issue Mar 19, 2014 · 24 comments

Comments

Projects
None yet
@djsly
Copy link

djsly commented Mar 19, 2014

I was using the following

ENV MYPATH /usr/local/myPath
CMD [ "ls", "-la", "$MYPATH"]

and I get
ls: cannot access $MYPATH: No such file or directory

But, if I used the other format
CMD ls -la $MYPATH

then the listing is correct.

CMD [ "ls", "-la", $NMMHOME ] simply doesn't work

Now I also was trying something like this.

ENTRYPOINT [ "$MYPATH/start.sh"]

doesn't work. Some problem : unable to locate $MYPATH/start.sh

But similarly to CMD 'shell version'

ENTRYPOINT $MYPATH/start.sh

works fine.

Also the start.sh is referencing the $MYPATH as well and it sees the correct value.

So I think there is still a bug in specifying the environment variables within the CMD / ENTRYPOINT commands when using the exec like way (preferred method)

@djsly

This comment has been minimized.

Copy link
Author

djsly commented Mar 20, 2014

Running 0.9.0, on RHEL6

@SvenDowideit

This comment has been minimized.

Copy link
Contributor

SvenDowideit commented Mar 21, 2014

Yes, The quoted ["$MYPATH"] style params for ENTRYPOINT and CMD are special, as they get started without a shell (so you can choose your own) and iirc they are escaped too.

@djsly

This comment has been minimized.

Copy link
Author

djsly commented Mar 21, 2014

Wouldn't there be a way to be able to resolve environment variable with the recommend CMD / ENTRYPOINT command ? to me this sounds like a bug... or at least a doc update should be required to avoid future confusion.

Fortunately, the non [" "," "] format still works :)

@tianon

This comment has been minimized.

Copy link
Member

tianon commented Mar 21, 2014

They're not escaped perse, but the shell itself is what expands $ENV_VARS, so if you're bypassing the shell, it doesn't really make sense for environment variables to be expanded.

@duglin

This comment has been minimized.

Copy link
Contributor

duglin commented Sep 11, 2014

From my testing it appears that:
CMD [ "/bin/bash", "-c", "ls -la $MYPATH"]
would work.
But, of course, that loses the ability to list each arg separately and at this point why even bother using the [ ] format of CMD at all?

For my own education... when should people use CMD [ ... ] vs CMD ... ?
Is it just when you don't want things to default to "sh -c" ? The docs push people towards the [ ] format, but why? It seems actually more limiting due to issues like the one being raised here.

@davealbert

This comment has been minimized.

Copy link

davealbert commented Feb 17, 2015

I too am having trouble understanding when to use the exec format CMD [...] versus the shell format CMD ... I understand the exec format is preferred, but I don't understand why because of the limitations.

@duglin

This comment has been minimized.

Copy link
Contributor

duglin commented Mar 4, 2015

@tiborvass I know that we explicitly don't do env var substitutions for CMD and ENTRYPOINT, with the assumption that the shell will do it, but I'm wondering if we should do it just when the JSON format is used? It won't be as full featured as the real shell (even with #11149 ) but it'll be better than what do today. Thoughts?

@tianon

This comment has been minimized.

Copy link
Member

tianon commented Mar 5, 2015

@tiborvass

This comment has been minimized.

Copy link
Collaborator

tiborvass commented Mar 7, 2015

I'll just say that if there is an easy workaround (and in this case there is), let's keep the workaround for now. It shouldn't be a big pain and if it is we'll know it.

Maybe we could make sure the documentation is very explicit about this difference?
Ping @SvenDowideit @moxiegirl @fredlf

Other than that, I'd be tempted to close this issue.

@moxiegirl

This comment has been minimized.

Copy link
Contributor

moxiegirl commented Mar 8, 2015

@tiborvass I think it makes sense to clarify this in the docs. I'll label it white-belt and if it doesn't get cleaned up by the birthday, I'll pick it up.

@duglin

This comment has been minimized.

Copy link
Contributor

duglin commented Mar 8, 2015

Just checked and I think the docs are probably ok, both CMD and ENTRYPOINT have text like this:


Note: Unlike the shell form, the exec form does not invoke a command shell. This means that normal shell processing does not happen. For example, ENTRYPOINT [ "echo", "$HOME" ] will not do variable substitution on $HOME. If you want shell processing then either use the shell form or execute a shell directly, for example: ENTRYPOINT [ "sh", "-c", "echo", "$HOME" ]. Variables that are defined in the Dockerfileusing ENV, will be substituted by the Dockerfile parser.


which I think covers it. But if you want it to say more let me know and I can tweak it.

@tiborvass

This comment has been minimized.

Copy link
Collaborator

tiborvass commented Mar 8, 2015

@duglin I think the confusion comes from the builder replacing variables, versus the shell replacing variables. The docs you referred to only addresses the shell substitution. Currently, we substitute VOLUME $foo as well as VOLUME ["$foo"] if foo was defined by an ENV. IOW there's $HOME from the shell, and there's $HOME if you used ENV HOME. That's where the confusion comes from.

Personal opinion: At this point I don't know what to do to make this clearer, we keep digging a bigger hole with these tiny changes that seem to be fine and relieve an immediate pain only to later introduce more confusion :(

I would vote for discouraging replacing variables in the JSON format, and that for all commands, but I am afraid there is no good solution right now.

@duglin

This comment has been minimized.

Copy link
Contributor

duglin commented Mar 8, 2015

@tiborvass there's also: https://docs.docker.com/reference/builder/#environment-replacement
IMO any confusion that might be there is when people expect it to act like it does on the shell, but sometimes doesn't, but we're getting closer. Its when we don't do what people expect, like substituting on the JSON form, that I think will confuse people - but I have a personal dislike for that form :-)

@cpuguy83

This comment has been minimized.

Copy link
Contributor

cpuguy83 commented Sep 15, 2015

Can we close this? This looks like well documented, expected behavior, unless you found somewhere the docs seem lacking?

@tiborvass

This comment has been minimized.

Copy link
Collaborator

tiborvass commented Sep 23, 2015

Agree with @cpuguy83. Closing.

Let us know if you think docs can improved. Feel free also to send a Pull Request to the documentation if you want to :)

@tiborvass tiborvass closed this Sep 23, 2015

@djsly

This comment has been minimized.

Copy link
Author

djsly commented Sep 23, 2015

Sorry for not responding before. This issue can be closed no problem. The doc is more clear indeed. 
Thanks ! 

Sent from Outlook on iPhone

On Tue, Sep 22, 2015 at 5:19 PM -0700, "Tibor Vass" notifications@github.com wrote:

Closed #4783.


Reply to this email directly or view it on GitHub.

@apapanico

This comment has been minimized.

Copy link

apapanico commented May 24, 2016

Hey, I just found this issue but I think it's pertinent. Does the environment variable behavior also apply when launching a container using an entrypoint script:

ENTRYPOINT ['./entrypoint.sh']

I've been running into this problem where the dockerfile sets an environment variable and it carries over when doing a debug launch with bash but not when doing the full launch with the entrypoint script. It seems likely it's the same thing.

If so, my struggle to understand this behavior for a while was I was looking under https://docs.docker.com/engine/reference/builder/#env and nothing there points out this issue of environment variables being lost because of the exec form of CMD and ENTRYPOINT. Perhaps a reference like this in the ENV section of the builder reference.

Note:
As noted for CMD and ENTRYPOINT, the exec form for CMD and ENTRYPOINT does not invoke
a command shell. This means that normal shell processing does not happen and therefore
variable substitution will not be applied. The consequence is that environment variables set here will
not available to the container when using the exec form.

@apapanico

This comment has been minimized.

Copy link

apapanico commented May 24, 2016

Actually, I suspect the variable substitution will never work with an ENTRYPOINT script. It seems that with both ENTRYPOINT and CMD, there is no variable substitution. Perhaps it defaults to exec form because of the combination? I found a workaround for myself.

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented May 24, 2016

Note that your example is not valid JSON;

ENTRYPOINT ['./entrypoint.sh']

Should be

ENTRYPOINT ["./entrypoint.sh"]

(Double quotes)

@duglin

This comment has been minimized.

Copy link
Contributor

duglin commented May 24, 2016

env var are available in your script/environment - they're just not processed on the cmd line since there's no shell to parse/process them.

@apapanico

This comment has been minimized.

Copy link

apapanico commented May 24, 2016

Okay. I had a few things wrong above. Here's a complete MWE that I think shows the full extent of the behavior. (And in practice I was using proper JSON. Just wrote it incorrectly above).

Each variable is meant to vary among user which created it (VAR_1 vs VAR_2), where it was created (VAR_1 and VAR_2 created in Dockerfile; VAR_3 in docker-compose.yml, and VAR_4 and VAR_5 in entrypoint.sh), and VAR_4 and VAR_5 with export or not.

The variables are present in the entrypoint script but none of them make it to the exec command (even the one that was exported). I would like to be able to have the variables available for that final exec command.

Dockerfile

FROM debian:wheezy
ENV VAR_1 var1
RUN useradd -ms /bin/bash -d /myuser/ myuser
COPY entrypoint.sh /myuser/entrypoint.sh 
RUN chmod +x /myuser/entrypoint.sh
RUN chown -R myuser:myuser /myuser
USER myuser
ENV VAR_2 var2
ENTRYPOINT ["/myuser/entrypoint.sh"]

entrypoint.sh

CMD="echo"
export VAR_4=var4
VAR_5=var5
echo "Inside entrypoint.sh:" ${VAR_1} ${VAR_2} ${VAR_3} ${VAR_4} ${VAR_5}
exec $CMD "$@"

docker-compose.yml

example:
    image: varexample
    restart: always
    environment:
      - VAR_3=var3
    command: "docker-compose command: ${VAR_1} ${VAR_2} ${VAR_3} ${VAR_4} ${VAR_5}"

Log output

$ docker-compose logs example
WARNING: The VAR_1 variable is not set. Defaulting to a blank string.
WARNING: The VAR_2 variable is not set. Defaulting to a blank string.
WARNING: The VAR_3 variable is not set. Defaulting to a blank string.
WARNING: The VAR_4 variable is not set. Defaulting to a blank string.
WARNING: The VAR_5 variable is not set. Defaulting to a blank string.
Attaching to dockerexample_example_1
example_1  | Inside entrypoint.sh: var1 var2 var3 var4 var5
example_1  | docker-compose command:
...
@mNantern

This comment has been minimized.

Copy link

mNantern commented Oct 11, 2016

I have the same problem as @apapanico . Is there a workaround or any news about that issue ?

Thank you !

@ORESoftware

This comment has been minimized.

@slaout

This comment has been minimized.

Copy link

slaout commented Nov 28, 2018

Let's say our Dockerfile has:

ENV VARIABLE=replaced

And we want to run this:

docker run <image> arg1 arg2

I obviously wanted this output:

# => replaced arg1 arg2

But no provided solution would allow both variable substitution and arguments passing.

Here are the options I tested (the one found on internet, plus a few others):

ENTRYPOINT echo $VARIABLE
# => replaced

ENTRYPOINT [ "echo", "$VARIABLE" ]
# => $VARIABLE arg1 arg2

ENTRYPOINT [ "sh", "-c", "echo", "$VARIABLE" ]
# =>

ENTRYPOINT [ "sh", "-c", "echo $VARIABLE" ]
# => replaced

ENTRYPOINT [ "sh", "-c", "echo $VARIABLE $@" ]
# => replaced arg2

ENTRYPOINT [ "sh", "-c", "echo $@", "$VARIABLE" ]
# => arg1 arg2

And FINALLY, I found this one:

ENTRYPOINT [ "sh", "-c", "echo $VARIABLE $0 $@" ]
# => replaced arg1 arg2

It works!!!
But I feel SOOOO dirty!

Obviously, in real life, I wanted to do something more useful:

docker run <image> --app.options.additional.second=true --app.options.additional.third=false

ENTRYPOINT [ "sh", "-c", "java -Xmx$XMX $0 $@", \
             "-jar", \
             "/my.jar", \
             "--app.options.first=true" ]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.