New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerfile: VOLUME declaration before COPY or ADD of multiple files or whole directory adds to the image #21728

Open
genei09 opened this Issue Apr 1, 2016 · 10 comments

Comments

Projects
None yet
8 participants
@genei09
Copy link

genei09 commented Apr 1, 2016

docker build and Dockerfile does not respect VOLUME if COPY or ADD move in multiple files or a whole directory. If a single file is specified in either COPY or ADD it works as expected. The file is available during the build and then not present in the image.


BUG REPORT INFORMATION

Output of docker version:


Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.3
 Git commit:   a34a1d5
 Built:        Fri Nov 20 17:56:04 UTC 2015
 OS/Arch:      darwin/amd64

Server:
 Version:      1.10.0
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   590d5108
 Built:        2016-02-04T19:55:25.696148927+00:00
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 2
Images: 5
Server Version: 1.10.0
Storage Driver: btrfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.3.3-dhyve
Operating System: Buildroot 2015.11.1
CPUs: 1
Total Memory: 3.86 GiB
Name: dlite
ID: MZRL:W4WC:2JDK:Z2JS:F5KM:UGUT:VFJV:ZO7M:ATVR:66HH:CS4L:5PXL

dlite 1.4.0
OSx 10.11.4

Steps to reproduce the issue:
0. make a dir "sample" with a text file "textfile.txt" with some contents

  1. Dockerfile using an OS base and with contents
    VOLUME /opt/sample
    COPY sample/ /opt/sample
  2. docker build volume_sample
  3. docker run -it volume_sample bash
  4. cat /opt/sample/textfile.txt

Describe the results you received:
the contents of textfile.txt are output

Describe the results you expected:
an error of "No such file or directory"

Additional information you deem important (e.g. issue happens only occasionally):

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Apr 1, 2016

/cc @cpuguy83

@programmerq programmerq added the kind/bug label Apr 3, 2016

@RyanMagnusson

This comment has been minimized.

Copy link

RyanMagnusson commented Apr 4, 2016

Is "No such file or directory" the correct error to return here?
Isn't the COPY command here working as specified and creating the folder structure correctly?

@senk

This comment has been minimized.

Copy link
Contributor

senk commented Apr 4, 2016

I think "No such file or directory" is the correct result to expect, at least according to https://docs.docker.com/engine/reference/builder/#volume which states "Note: If any build steps change the data within the volume after it has been declared, those changes will be discarded."

Sample Dockerfile:

FROM ubuntu
VOLUME /opt/sample
COPY sample/ /opt/sample
RUN /bin/echo 'testfail' >> /opt/sample/textfile.txt
RUN /bin/echo 'testfail' >> /textfile.txt

In this case RUN /bin/echo 'testfail' >> /opt/sample/textfile.txt is correctly discarded as the note above states, but the COPY Build Step should be discarded as well if "any build steps change the data within the volume after it has been declared, those changes will be discarded."

@genei09

This comment has been minimized.

Copy link
Author

genei09 commented Apr 4, 2016

It's also worth noting that the changes ARE discarded if you only COPY|ADD a single file

@senk

This comment has been minimized.

Copy link
Contributor

senk commented Apr 4, 2016

@genei09 do you have an example? i was not able to reproduce that. for me no copy or add (single file as well as folders) ever gets discarded

@genei09

This comment has been minimized.

Copy link
Author

genei09 commented Apr 4, 2016

Same as you have above with:
COPY sample/testfile.txt /opt/sample/testpass.txt

I should have said "when I specify a single file"

This has worked for me putting an rsa key into a build temporarily for example

@senk

This comment has been minimized.

Copy link
Contributor

senk commented Apr 4, 2016

single files aren't discarded for me:

root@docker:~/dockertest# docker build -t volume_sample .
Sending build context to Docker daemon 4.608 kB
Step 1 : FROM ubuntu
 ---> 07c86167cdc4
Step 2 : VOLUME /opt/sample
 ---> Running in acb60088b165
 ---> e832c00d057a
Removing intermediate container acb60088b165
Step 3 : COPY sample/textfile.txt /opt/sample/textfile.txt
 ---> d355e38ed64a
Removing intermediate container a7fcbf8b23a3
Step 4 : RUN /bin/echo 'testfail' >> /opt/sample/textfile.txt
 ---> Running in 30667b4f7c3b
 ---> ba938650e9f6
Removing intermediate container 30667b4f7c3b
Step 5 : RUN /bin/echo 'testfail' >> /textfile.txt
 ---> Running in 14f4f14aa91b
 ---> 12b84241f88f
Removing intermediate container 14f4f14aa91b
Successfully built 12b84241f88f
root@docker:~/dockertest# docker run -it volume_sample bash
root@5d1196c02b63:/# cat /opt/sample/textfile.txt
test
@RyanMagnusson

This comment has been minimized.

Copy link

RyanMagnusson commented Apr 4, 2016

Perhaps this is a documentation issue then, because I think this works as I've come to expect it.

IIRC volume and copy commands execute at different layers, since volumes exist outside the container they mounted outside everything else. In this scenario, since we aren't binding anything to this path it appears to just pass through to what was written inside the container.

IMHO the documentation is stating that if you mount a container, and think you can copy extra files or changes to it -- those will be ignored because you really cannot do anything to the volume until the container is actually created.

When you execute docker run and create a new container without binding a volume to it then the copied directories & files remain (which is what the example scenario is).
However if you pass a volume -v [host]:[container] then the changes are discarded.

Is the expected behavior that any ADD, COPY, or RUN command should be ignored if a previous VOLUME command is performed?

@genei09

This comment has been minimized.

Copy link
Author

genei09 commented Apr 4, 2016

@senk How strange, I'm not able to repeat the single file method anymore. I'd swear I was able to get it to work at one point.

@RyanMagnusson

It seems rather counter intuitive that

VOLUME /user/.ssh
COPY id_rsa /user/.ssh/
RUN ssh-keyscan -t rsa github.com 2>&1 > /user/.ssh/known_hosts

results in /user/.ssh having only id_rsa in it while:

RUN ssh-keyscan -t rsa github.com 2>&1 > /user/.ssh/known_hosts
VOLUME /user/.ssh
COPY id_rsa /user/.ssh/

results in both files being in /user/.ssh/

It makes it very difficult to get temporary scripts and source files into a build without bloating the image or exposing rsa keys. The best alternative is to use docker run and commit when you're done.

@cpuguy83

This comment has been minimized.

Copy link
Contributor

cpuguy83 commented Apr 4, 2016

The behavior is unintentional, but I don't think it's something we can change.
Definitely warrants a doc change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment