New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Heredoc (or multi line instructions) in RUN command (in Dockerfile) #1554

Closed
brikou opened this Issue Aug 15, 2013 · 41 comments

Comments

Projects
None yet
@brikou

brikou commented Aug 15, 2013

An interesting use case would be to patch config file within dockerfile

RUN cat << EOF | patch --strip 0
--- /etc/apache2/conf.d/charset
--- /etc/apache2/conf.d/charset
@@ -3,4 +3,4 @@
 # have this encoding. It will override any encoding given in the files
 # in meta http-equiv or xml encoding tags.

-#AddDefaultCharset UTF-8
+AddDefaultCharset UTF-8
EOF
@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Aug 15, 2013

Collaborator

To focus on your specific example: it might make more sense to commit your config & dockerfile to a git or mercurial repository, and ADD the config file during the build.

  ​

   



—

@solomonstre
@getdocker

On Thu, Aug 15, 2013 at 7:59 AM, Brikou CARRE notifications@github.com
wrote:

An interesting example would be to patch config file with dockerfile

RUN cat << EOF | patch --strip 0
--- /etc/apache2/conf.d/charset
--- /etc/apache2/conf.d/charset
@@ -3,4 +3,4 @@
 # have this encoding. It will override any encoding given in the files
 # in meta http-equiv or xml encoding tags.
-#AddDefaultCharset UTF-8
+AddDefaultCharset UTF-8
EOF

Reply to this email directly or view it on GitHub:
#1554

Collaborator

shykes commented Aug 15, 2013

To focus on your specific example: it might make more sense to commit your config & dockerfile to a git or mercurial repository, and ADD the config file during the build.

  ​

   



—

@solomonstre
@getdocker

On Thu, Aug 15, 2013 at 7:59 AM, Brikou CARRE notifications@github.com
wrote:

An interesting example would be to patch config file with dockerfile

RUN cat << EOF | patch --strip 0
--- /etc/apache2/conf.d/charset
--- /etc/apache2/conf.d/charset
@@ -3,4 +3,4 @@
 # have this encoding. It will override any encoding given in the files
 # in meta http-equiv or xml encoding tags.
-#AddDefaultCharset UTF-8
+AddDefaultCharset UTF-8
EOF

Reply to this email directly or view it on GitHub:
#1554

@unclejack

This comment has been minimized.

Show comment
Hide comment
@unclejack

unclejack Sep 8, 2013

Contributor

I'm closing this in favor of #1799. If you think this issue should be re-opened, please re-open it or ask me to do it.

Contributor

unclejack commented Sep 8, 2013

I'm closing this in favor of #1799. If you think this issue should be re-opened, please re-open it or ask me to do it.

@unclejack unclejack closed this Sep 8, 2013

@d11wtq

This comment has been minimized.

Show comment
Hide comment
@d11wtq

d11wtq Mar 13, 2014

The problem with using a separate file + ADD is that it will be cached, so you can't tweak what would have been in the heredoc. You have to completely avoid caching everything that comes before the multi-line part.

If you actually want to use heredoc, as is really common in the case of the pattern:

cat > /etc/some/file.conf <<'EOF'
Whatever multi-line stuff
I need to put here
without any headaches
EOF

Then using the \ syntax, Docker squashes it into a single line and breaks the heredoc.

d11wtq commented Mar 13, 2014

The problem with using a separate file + ADD is that it will be cached, so you can't tweak what would have been in the heredoc. You have to completely avoid caching everything that comes before the multi-line part.

If you actually want to use heredoc, as is really common in the case of the pattern:

cat > /etc/some/file.conf <<'EOF'
Whatever multi-line stuff
I need to put here
without any headaches
EOF

Then using the \ syntax, Docker squashes it into a single line and breaks the heredoc.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Mar 13, 2014

Member

ADD does cache though, for several versions now

Member

tianon commented Mar 13, 2014

ADD does cache though, for several versions now

@d11wtq

This comment has been minimized.

Show comment
Hide comment
@d11wtq

d11wtq Mar 13, 2014

Yes, that is a problem. I want to be able to edit the file and have docker do "smart" caching. If the string that is in this file was part of a command, docker would understand it can't use the cache. But put it in a file and suddenly the cache effectively makes the file immutable, unless you completely go around the caching, which in my case (entire llvm toolchain) I don't want to do.

d11wtq commented Mar 13, 2014

Yes, that is a problem. I want to be able to edit the file and have docker do "smart" caching. If the string that is in this file was part of a command, docker would understand it can't use the cache. But put it in a file and suddenly the cache effectively makes the file immutable, unless you completely go around the caching, which in my case (entire llvm toolchain) I don't want to do.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Mar 14, 2014

Member

Previously, ADD was just always a cache bust, but now it is as you might expect, busting based on a checksum of what was added, so changing the file will cause a natural cache bust as you might expect.

Member

tianon commented Mar 14, 2014

Previously, ADD was just always a cache bust, but now it is as you might expect, busting based on a checksum of what was added, so changing the file will cause a natural cache bust as you might expect.

@breerly

This comment has been minimized.

Show comment
Hide comment
@breerly

breerly commented Jun 14, 2014

+1 here

@gnutix

This comment has been minimized.

Show comment
Hide comment
@gnutix

gnutix commented Jun 23, 2014

+1 !

@ianbytchek

This comment has been minimized.

Show comment
Hide comment
@ianbytchek

ianbytchek commented Nov 15, 2014

+10000000

@ailjushkin

This comment has been minimized.

Show comment
Hide comment
@ailjushkin

ailjushkin Feb 2, 2015

How can I run something from a user different from the root user when building new image ?

ailjushkin commented Feb 2, 2015

How can I run something from a user different from the root user when building new image ?

@unclejack

This comment has been minimized.

Show comment
Hide comment
@unclejack

unclejack Feb 2, 2015

Contributor

@PositiveAlex You can use USER someuser after adding that user.

Contributor

unclejack commented Feb 2, 2015

@PositiveAlex You can use USER someuser after adding that user.

@ailjushkin

This comment has been minimized.

Show comment
Hide comment
@ailjushkin

ailjushkin Feb 3, 2015

@unclejack thank you, and sorry, I should read manual more accurate first...

ailjushkin commented Feb 3, 2015

@unclejack thank you, and sorry, I should read manual more accurate first...

@zzeroo

This comment has been minimized.

Show comment
Hide comment
@zzeroo

zzeroo Aug 20, 2015

No Heredoc and a little ugly but it work ...

RUN echo " \n\
KERNEL=="mali", MODE="0660", GROUP="video" \n\
KERNEL=="ump", MODE="0660", GROUP="video" \n\
" >> /etc/udev/rules.d/50-mali.rules

+1 Heredoc

zzeroo commented Aug 20, 2015

No Heredoc and a little ugly but it work ...

RUN echo " \n\
KERNEL=="mali", MODE="0660", GROUP="video" \n\
KERNEL=="ump", MODE="0660", GROUP="video" \n\
" >> /etc/udev/rules.d/50-mali.rules

+1 Heredoc

@GordonTheTurtle

This comment has been minimized.

Show comment
Hide comment
@GordonTheTurtle

GordonTheTurtle Sep 9, 2015

USER POLL

The best way to get notified when there are changes in this discussion is by clicking the Subscribe button in the top right.

The people listed below have appreciated your meaningfull discussion with a random +1:

@sleaze
@MatthewRalston
@dsissitka
@mingfang
@grigio
@devinrsmith
@afolarin
@ggershoni

GordonTheTurtle commented Sep 9, 2015

USER POLL

The best way to get notified when there are changes in this discussion is by clicking the Subscribe button in the top right.

The people listed below have appreciated your meaningfull discussion with a random +1:

@sleaze
@MatthewRalston
@dsissitka
@mingfang
@grigio
@devinrsmith
@afolarin
@ggershoni

@xasm83

This comment has been minimized.

Show comment
Hide comment
@xasm83

xasm83 commented Sep 15, 2015

+1

@ryanmaclean

This comment has been minimized.

Show comment
Hide comment
@ryanmaclean

ryanmaclean Nov 24, 2015

Would definitely be interested in seeing this as well! 👍

ryanmaclean commented Nov 24, 2015

Would definitely be interested in seeing this as well! 👍

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Nov 24, 2015

Member

Also see this comment; #16058 (comment)

Member

thaJeztah commented Nov 24, 2015

Also see this comment; #16058 (comment)

@nimkar

This comment has been minimized.

Show comment
Hide comment
@nimkar

nimkar Jan 28, 2016

+1 for Heredoc

nimkar commented Jan 28, 2016

+1 for Heredoc

@j4zzcat

This comment has been minimized.

Show comment
Hide comment
@j4zzcat

j4zzcat commented Feb 2, 2016

+1

4 similar comments
@eliezio

This comment has been minimized.

Show comment
Hide comment
@eliezio

eliezio commented Mar 3, 2016

+1

@sergeyklay

This comment has been minimized.

Show comment
Hide comment
@sergeyklay

sergeyklay commented Mar 4, 2016

+1

@geekaholic

This comment has been minimized.

Show comment
Hide comment
@geekaholic

geekaholic commented Mar 10, 2016

+1

@cjac

This comment has been minimized.

Show comment
Hide comment
@cjac

cjac commented Mar 12, 2016

+1

@C-Duv

This comment has been minimized.

Show comment
Hide comment
@C-Duv

C-Duv Mar 12, 2016

Stop posting +1, use the new GitHub reactions feature.

C-Duv commented Mar 12, 2016

Stop posting +1, use the new GitHub reactions feature.

@sleaze

This comment has been minimized.

Show comment
Hide comment
@sleaze

sleaze Mar 14, 2016

+1 Thank you! So glad and happy this got implemented!

sleaze commented Mar 14, 2016

+1 Thank you! So glad and happy this got implemented!

@djmaze

This comment has been minimized.

Show comment
Hide comment
@djmaze

djmaze Mar 14, 2016

Contributor

@cjac You referenced an old PR which implemented the \ multiline syntax. The Heredoc syntax has not been implemented yet. (And that's what this issue is now about.) It seems the Dockerfile syntax is currently frozen, so this probably won't be possible before Docker 2.0 (or something like that..).

Contributor

djmaze commented Mar 14, 2016

@cjac You referenced an old PR which implemented the \ multiline syntax. The Heredoc syntax has not been implemented yet. (And that's what this issue is now about.) It seems the Dockerfile syntax is currently frozen, so this probably won't be possible before Docker 2.0 (or something like that..).

@sleaze

This comment has been minimized.

Show comment
Hide comment
@sleaze

sleaze Mar 15, 2016

Oh :/ sad now

sleaze commented Mar 15, 2016

Oh :/ sad now

@junjiemars

This comment has been minimized.

Show comment
Hide comment
@junjiemars

junjiemars Aug 7, 2016

Heredoc is awesome for writing small script/config, etc. I don't care what's cache it or not.

junjiemars commented Aug 7, 2016

Heredoc is awesome for writing small script/config, etc. I don't care what's cache it or not.

@denysvitali

This comment has been minimized.

Show comment
Hide comment
@denysvitali

denysvitali Nov 21, 2016

👍 This should be implemented!

denysvitali commented Nov 21, 2016

👍 This should be implemented!

@mingfang

This comment has been minimized.

Show comment
Hide comment
@mingfang

mingfang Nov 22, 2016

+1 just to piss off @GordonTheTurtle

mingfang commented Nov 22, 2016

+1 just to piss off @GordonTheTurtle

@verbunk

This comment has been minimized.

Show comment
Hide comment
@verbunk

verbunk Apr 11, 2017

This thread is a bit dated now but I still haven't found native support for this. Anyhoo, in case it helps someone else what I do is pull alpine, add in coreutils as a virtual-dep, move base64 to /usr/local/bin and in my Dockerfile, RUN echo "" | base64 --decode > /etc/myconf.conf so it will copy over my base configs. Supports line breaks in the encoded form so no mess (if you consider this non-messy). Extra points if you use envsubst (add as virtual-dep like above, then move before deleting the large unnecessary packages) to mod the config file with your ENV variables.

I just really hate extra files hanging around.

verbunk commented Apr 11, 2017

This thread is a bit dated now but I still haven't found native support for this. Anyhoo, in case it helps someone else what I do is pull alpine, add in coreutils as a virtual-dep, move base64 to /usr/local/bin and in my Dockerfile, RUN echo "" | base64 --decode > /etc/myconf.conf so it will copy over my base configs. Supports line breaks in the encoded form so no mess (if you consider this non-messy). Extra points if you use envsubst (add as virtual-dep like above, then move before deleting the large unnecessary packages) to mod the config file with your ENV variables.

I just really hate extra files hanging around.

@ChipmunkV

This comment has been minimized.

Show comment
Hide comment
@ChipmunkV

ChipmunkV Apr 28, 2017

One possible workaround is the following:

bash -c "$(/bin/echo -e "cat > /etc/my.config <<EOM\
\n########################################################\
\n# The file\
\n########################################################\
\na = b\
\nEOM\n")

Prefix everything with \n and suffix with \. The echo evaluates newlines for bash.

ChipmunkV commented Apr 28, 2017

One possible workaround is the following:

bash -c "$(/bin/echo -e "cat > /etc/my.config <<EOM\
\n########################################################\
\n# The file\
\n########################################################\
\na = b\
\nEOM\n")

Prefix everything with \n and suffix with \. The echo evaluates newlines for bash.

@sleaze

This comment has been minimized.

Show comment
Hide comment
@sleaze

sleaze Jan 24, 2018

+1

BUMP still waiting on this one ......

Cheerio

sleaze commented Jan 24, 2018

+1

BUMP still waiting on this one ......

Cheerio

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah
Member

thaJeztah commented Jan 24, 2018

@ktf

This comment has been minimized.

Show comment
Hide comment
@ktf

ktf commented May 16, 2018

+1

2 similar comments
@sharkguto

This comment has been minimized.

Show comment
Hide comment
@sharkguto

sharkguto commented Sep 13, 2018

+1

@mkomadel

This comment has been minimized.

Show comment
Hide comment
@mkomadel

mkomadel commented Oct 4, 2018

+1

@brikou

This comment has been minimized.

Show comment
Hide comment
@brikou

brikou Oct 4, 2018

@unclejack 5 years ago you told me you can reopen? Can you?

brikou commented Oct 4, 2018

@unclejack 5 years ago you told me you can reopen? Can you?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Oct 4, 2018

Member

There is a new issue / proposal that I linked to; #34423. Please continue the conversation over there (also, please don't add +1's - it generates a lot of noise, and distracts from the actual discussion)

Member

thaJeztah commented Oct 4, 2018

There is a new issue / proposal that I linked to; #34423. Please continue the conversation over there (also, please don't add +1's - it generates a lot of noise, and distracts from the actual discussion)

@brikou

This comment has been minimized.

Show comment
Hide comment
@brikou

brikou Oct 4, 2018

@thaJeztah your proposal is nice but really not the same as not dealing with shell heredoc.

brikou commented Oct 4, 2018

@thaJeztah your proposal is nice but really not the same as not dealing with shell heredoc.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Oct 4, 2018

Member

@brikou I think it should allow you to do that, e.g. something like this;

RUN <<EOF
cat > /etc/my.config <<EOM
########################################################
# The file
########################################################
a = b
EOM
EOF
Member

thaJeztah commented Oct 4, 2018

@brikou I think it should allow you to do that, e.g. something like this;

RUN <<EOF
cat > /etc/my.config <<EOM
########################################################
# The file
########################################################
a = b
EOM
EOF
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment