New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving a file from the host system to a container via docker cp #5846

Closed
jamtur01 opened this Issue May 16, 2014 · 106 comments

Comments

Projects
None yet
@jamtur01
Contributor

jamtur01 commented May 16, 2014

The docker cp command can currently copy a file from the container to the host. It should also be able to do the reverse.

Ping @crosbymichael

@hadim

This comment has been minimized.

Show comment
Hide comment
@hadim

hadim commented May 16, 2014

+1

@bruno-

This comment has been minimized.

Show comment
Hide comment
@bruno-

bruno- commented May 23, 2014

+1

@maci0

This comment has been minimized.

Show comment
Hide comment
@maci0

maci0 commented May 24, 2014

+1

@LK4D4

This comment has been minimized.

Show comment
Hide comment
@LK4D4

LK4D4 May 26, 2014

Contributor

+1

Contributor

LK4D4 commented May 26, 2014

+1

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael May 27, 2014

Contributor

This there a reason why this cannot be accomplished with an ADD in the Dockerfile?

Contributor

crosbymichael commented May 27, 2014

This there a reason why this cannot be accomplished with an ADD in the Dockerfile?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 27, 2014

Member

@crosbymichael I think the difference here is that docker cp can be used on a running container. Since this command allows copying from a container to the host, allowing it to work in the other direction makes sense (and is something a user will expect)

Interactively adding/updating a file in a running container can be very convenient, other options (running sshd, volumes, FTP?) could work, but won't be appropriate in many cases.

Member

thaJeztah commented May 27, 2014

@crosbymichael I think the difference here is that docker cp can be used on a running container. Since this command allows copying from a container to the host, allowing it to work in the other direction makes sense (and is something a user will expect)

Interactively adding/updating a file in a running container can be very convenient, other options (running sshd, volumes, FTP?) could work, but won't be appropriate in many cases.

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael May 27, 2014

Contributor

Do you have an example of some file you would copy into a running container?

Contributor

crosbymichael commented May 27, 2014

Do you have an example of some file you would copy into a running container?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 28, 2014

Member

(Trying to provide a reasonable example here; at this moment I'm using docker for development-environments and use volumes to share files between the host and container)

I'm thinking of configuration-files that may change during the lifetime of a container; sometimes you don't want to shutdown or re-build a container to change that.

@crosbymichael Is it ok if I reverse the question? Are there any particular reasons to not implement copying to the container?

The point is, that at this moment there is (afaik) no way to 'enter' a container to make small changes (interactively) without installing a service for that (e.g. sshd). Docker 'enter' (nsenter?) isn't ready yet (or at least only considered for debugging).

Member

thaJeztah commented May 28, 2014

(Trying to provide a reasonable example here; at this moment I'm using docker for development-environments and use volumes to share files between the host and container)

I'm thinking of configuration-files that may change during the lifetime of a container; sometimes you don't want to shutdown or re-build a container to change that.

@crosbymichael Is it ok if I reverse the question? Are there any particular reasons to not implement copying to the container?

The point is, that at this moment there is (afaik) no way to 'enter' a container to make small changes (interactively) without installing a service for that (e.g. sshd). Docker 'enter' (nsenter?) isn't ready yet (or at least only considered for debugging).

@lorin

This comment has been minimized.

Show comment
Hide comment
@lorin

lorin May 28, 2014

@crosbymichael I was thinking about implementing an Ansible connection plugin for Docker. Currently, if you want to use Ansible to configure a Docker container, you need to run an ssh server insde the container or you need to install Ansible inside of the container itself. With a connection plugin, you wouldn't need to do either of these things.

To implement an Ansible connection plugin, you need to be able to:

  • run an arbitrary command
  • copy an arbitrary file into the container
  • copy an arbitrary file out of the container

This feature is the missing element. I know this can be worked around using docker -v ... run cp ..., but having native support for copying files into containers would make the implementation cleaner.

lorin commented May 28, 2014

@crosbymichael I was thinking about implementing an Ansible connection plugin for Docker. Currently, if you want to use Ansible to configure a Docker container, you need to run an ssh server insde the container or you need to install Ansible inside of the container itself. With a connection plugin, you wouldn't need to do either of these things.

To implement an Ansible connection plugin, you need to be able to:

  • run an arbitrary command
  • copy an arbitrary file into the container
  • copy an arbitrary file out of the container

This feature is the missing element. I know this can be worked around using docker -v ... run cp ..., but having native support for copying files into containers would make the implementation cleaner.

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael May 28, 2014

Contributor

@thaJeztah The reason why i dont want to implement this is be cause ADD works well and can be reproduce to make image over a over. It's not one off command the modifies the container state.

Contributor

crosbymichael commented May 28, 2014

@thaJeztah The reason why i dont want to implement this is be cause ADD works well and can be reproduce to make image over a over. It's not one off command the modifies the container state.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 28, 2014

Member

@crosbymichael I think I get the point; in some perspective, containers should be built once (preferably via a Dockerfile) and regarded an immutable, deployable 'package'. To make changes, the container should be rebuilt and committed.

While this is a reasonable assumption for production purposes (and for deployment), my use case is a bit different;

We use dockers to create development environment/sandboxes. Specifications of those environments are not always pre-defined and may need some tweaking / modifications. Having to stop/rebuild a container for each change (to make use of ADD) seems a bit silly. Installing a SSH server on each container feels like too much overhead (but is our current approach, combined with volumes for source files) and lxc-attach is no longer an option with the native driver.

Being able to use cp to interactively add files is not a "must-have", but I'm silently hoping that some features will be added to allow 'interactive' sessions with a container; i.e. Transferring data to/from the host and entering the container. (I think work was in progress to add a docker enter command as replacement for lxc-attach?)

I hope my information is useful!

Member

thaJeztah commented May 28, 2014

@crosbymichael I think I get the point; in some perspective, containers should be built once (preferably via a Dockerfile) and regarded an immutable, deployable 'package'. To make changes, the container should be rebuilt and committed.

While this is a reasonable assumption for production purposes (and for deployment), my use case is a bit different;

We use dockers to create development environment/sandboxes. Specifications of those environments are not always pre-defined and may need some tweaking / modifications. Having to stop/rebuild a container for each change (to make use of ADD) seems a bit silly. Installing a SSH server on each container feels like too much overhead (but is our current approach, combined with volumes for source files) and lxc-attach is no longer an option with the native driver.

Being able to use cp to interactively add files is not a "must-have", but I'm silently hoping that some features will be added to allow 'interactive' sessions with a container; i.e. Transferring data to/from the host and entering the container. (I think work was in progress to add a docker enter command as replacement for lxc-attach?)

I hope my information is useful!

@apatil

This comment has been minimized.

Show comment
Hide comment
@apatil

apatil Jun 3, 2014

Contributor

@lorin +1 to the ansible plugin.

Contributor

apatil commented Jun 3, 2014

@lorin +1 to the ansible plugin.

@divoxx

This comment has been minimized.

Show comment
Hide comment
@divoxx

divoxx commented Jun 12, 2014

+1

@schickling

This comment has been minimized.

Show comment
Hide comment
@schickling

schickling commented Jun 18, 2014

+1

@ahazem

This comment has been minimized.

Show comment
Hide comment
@ahazem

ahazem Jun 22, 2014

@lorin +1 for the ansible plugin as well.

ahazem commented Jun 22, 2014

@lorin +1 for the ansible plugin as well.

@carltonbrown

This comment has been minimized.

Show comment
Hide comment
@carltonbrown

carltonbrown Jul 29, 2014

+1. Begging please for this. Sometimes I just need to add one tiny file to learn how to make a container work... almost REPL-like functionality if you think about it.

carltonbrown commented Jul 29, 2014

+1. Begging please for this. Sometimes I just need to add one tiny file to learn how to make a container work... almost REPL-like functionality if you think about it.

@jaloren

This comment has been minimized.

Show comment
Hide comment
@jaloren

jaloren commented Jul 30, 2014

+1

@jamshid

This comment has been minimized.

Show comment
Hide comment
@jamshid

jamshid Jul 30, 2014

Contributor

without this feature, people resort to copying into /var/lib/docker/aufs/mnt/ directly (eg, http://stackoverflow.com/questions/22907231/copying-files-from-host-to-docker-container)

Contributor

jamshid commented Jul 30, 2014

without this feature, people resort to copying into /var/lib/docker/aufs/mnt/ directly (eg, http://stackoverflow.com/questions/22907231/copying-files-from-host-to-docker-container)

@timfayz

This comment has been minimized.

Show comment
Hide comment
@timfayz

timfayz Aug 2, 2014

+1 highly support

timfayz commented Aug 2, 2014

+1 highly support

@coco98

This comment has been minimized.

Show comment
Hide comment
@coco98

coco98 Aug 8, 2014

+1 Need to copy SSH keys temporarily to the container to clone a private repo.

coco98 commented Aug 8, 2014

+1 Need to copy SSH keys temporarily to the container to clone a private repo.

@brunoric

This comment has been minimized.

Show comment
Hide comment
@brunoric

brunoric Aug 12, 2014

+1 on that.

brunoric commented Aug 12, 2014

+1 on that.

@l3iggs

This comment has been minimized.

Show comment
Hide comment
@l3iggs

l3iggs Aug 14, 2014

+1, this is my number one feature request for docker

l3iggs commented Aug 14, 2014

+1, this is my number one feature request for docker

@docbill

This comment has been minimized.

Show comment
Hide comment
@docbill

docbill Aug 14, 2014

For example, I have a docker container that has taken the place of a server I use to run a reporting server. For the most part it works well. The pain in the neck though is each time users run they need to add configuration files. I have one container container for oracle-xe where I used a volume. This works well for oracle-xe I'm happy just setting up one folder and using with every container made from that image. However, it is not an adequate solution for my reporting tool server. For the reporting tool, I really don't want copying data onto one container to impact another container. So that means creating a new folder for every container, doing a chattr on that folder so SELinux allows it to be used as a volume, keeping track what folder is mapped to which container, and clean-up of those folders when they are no longer used. Yuck!

Currently I use scp for the reporting tool. From within the container I scp to and from 172.17.41.1. That works, but is again a pain as it means repeatedly typing in my login credentials, as the container doesn't even know what user is running it, let alone that user's private keys. This approach means I can never use the container in a non-interactive mode...

"docker cp" offers a great alternative for copying data out of the container, but there really is no good way to get the data into the container. Certainly I could create a new image each time. But that means each and every time I create a new container, I also have to create a new Dockerfile, do a new build, and clean-up the new image when I'm done. Yuck. Certainly I could write a script to do all of this. But then since the script is not part of the container, it breaks the pattern that I should just be able to run my image everywhere.

Currently if I don't want to use scp, I can run my image anywhere, after manually installing and hacking a bunch of scripts to maintain per container images or volumes.

docbill commented Aug 14, 2014

For example, I have a docker container that has taken the place of a server I use to run a reporting server. For the most part it works well. The pain in the neck though is each time users run they need to add configuration files. I have one container container for oracle-xe where I used a volume. This works well for oracle-xe I'm happy just setting up one folder and using with every container made from that image. However, it is not an adequate solution for my reporting tool server. For the reporting tool, I really don't want copying data onto one container to impact another container. So that means creating a new folder for every container, doing a chattr on that folder so SELinux allows it to be used as a volume, keeping track what folder is mapped to which container, and clean-up of those folders when they are no longer used. Yuck!

Currently I use scp for the reporting tool. From within the container I scp to and from 172.17.41.1. That works, but is again a pain as it means repeatedly typing in my login credentials, as the container doesn't even know what user is running it, let alone that user's private keys. This approach means I can never use the container in a non-interactive mode...

"docker cp" offers a great alternative for copying data out of the container, but there really is no good way to get the data into the container. Certainly I could create a new image each time. But that means each and every time I create a new container, I also have to create a new Dockerfile, do a new build, and clean-up the new image when I'm done. Yuck. Certainly I could write a script to do all of this. But then since the script is not part of the container, it breaks the pattern that I should just be able to run my image everywhere.

Currently if I don't want to use scp, I can run my image anywhere, after manually installing and hacking a bunch of scripts to maintain per container images or volumes.

@jaloren

This comment has been minimized.

Show comment
Hide comment
@jaloren

jaloren Aug 16, 2014

From my point of view, the feature addresses a huge usability issue with docker builds: prototyping sucks.And prototyping is a really important part of the image design process for docker. The best way to find out is trying it out. I am not even a little bit interested going through an entire docker build JUST because my prototyping requires copying over a handful of configuration files or a script that I didn't realize I needed 5 minutes ago. Like others have mentioned we've hacked around this by using ssh and scp but that's way overkill.

allowing me to docker cp arbitrary files into a running container neatly addresses that concern.

jaloren commented Aug 16, 2014

From my point of view, the feature addresses a huge usability issue with docker builds: prototyping sucks.And prototyping is a really important part of the image design process for docker. The best way to find out is trying it out. I am not even a little bit interested going through an entire docker build JUST because my prototyping requires copying over a handful of configuration files or a script that I didn't realize I needed 5 minutes ago. Like others have mentioned we've hacked around this by using ssh and scp but that's way overkill.

allowing me to docker cp arbitrary files into a running container neatly addresses that concern.

@timhaak

This comment has been minimized.

Show comment
Hide comment
@timhaak

timhaak Aug 16, 2014

Agree that the main time this is needed is during prototyping. You have your container running and you quickly need to add something while trying to get the build correct.

Right now you would have to add ssh and have that started and then have it removed from the final build.

This I believe falls into the same space as nsenter.

timhaak commented Aug 16, 2014

Agree that the main time this is needed is during prototyping. You have your container running and you quickly need to add something while trying to get the build correct.

Right now you would have to add ssh and have that started and then have it removed from the final build.

This I believe falls into the same space as nsenter.

@comron

This comment has been minimized.

Show comment
Hide comment
@comron

comron Aug 19, 2014

+1 as well. For me I need the same thing as @tanmaig, I want to copy a temporary ssh key into the container that isn't known when the container image is built.

comron commented Aug 19, 2014

+1 as well. For me I need the same thing as @tanmaig, I want to copy a temporary ssh key into the container that isn't known when the container image is built.

@coco98

This comment has been minimized.

Show comment
Hide comment
@coco98

coco98 Aug 20, 2014

@comron The workaround I figured is that you just mount a data volume from the host to the computer. Then run SSH from the host computer etc, and copy data/git clone into the volume and it'll be available in the docker container. I guess the key idea behind not having the 'cp from host to container' is to force the stateless-ness of a docker.

coco98 commented Aug 20, 2014

@comron The workaround I figured is that you just mount a data volume from the host to the computer. Then run SSH from the host computer etc, and copy data/git clone into the volume and it'll be available in the docker container. I guess the key idea behind not having the 'cp from host to container' is to force the stateless-ness of a docker.

@tobegit3hub

This comment has been minimized.

Show comment
Hide comment
@tobegit3hub

tobegit3hub May 21, 2015

The workaround way by docker exec is inefficient when the file is large.

+1 for the PR to extend docker cp.

tobegit3hub commented May 21, 2015

The workaround way by docker exec is inefficient when the file is large.

+1 for the PR to extend docker cp.

@docbill

This comment has been minimized.

Show comment
Hide comment
@docbill

docbill May 21, 2015

It looks like #13171 is likely to be approved. One of the things that is problematic with the docker exec solution, is I find sometimes docker exec kills the container when exiting. As such is best avoided for running services.

docbill commented May 21, 2015

It looks like #13171 is likely to be approved. One of the things that is problematic with the docker exec solution, is I find sometimes docker exec kills the container when exiting. As such is best avoided for running services.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 May 21, 2015

Contributor

@docbill docker exec should not kill a container upon exiting (note, I'm +1 for the PR to add the docker cp functionality).

Contributor

cpuguy83 commented May 21, 2015

@docbill docker exec should not kill a container upon exiting (note, I'm +1 for the PR to add the docker cp functionality).

@alzarei

This comment has been minimized.

Show comment
Hide comment
@alzarei

alzarei May 25, 2015

+1

This is a very much appreciated feature for development. Although it is not the best practice for deployment! Please let the experienced users choose themselves!!

alzarei commented May 25, 2015

+1

This is a very much appreciated feature for development. Although it is not the best practice for deployment! Please let the experienced users choose themselves!!

@artjomzab

This comment has been minimized.

Show comment
Hide comment
@artjomzab

artjomzab Jun 2, 2015

  • 1

e.g. use case: copy mongo dump from host to mongo container in order to restore the data

artjomzab commented Jun 2, 2015

  • 1

e.g. use case: copy mongo dump from host to mongo container in order to restore the data

@ebuchman

This comment has been minimized.

Show comment
Hide comment
@ebuchman

ebuchman Jun 12, 2015

+1

This would especially be useful for data-only containers, where you might want to copy in a new configuration or private key.

ebuchman commented Jun 12, 2015

+1

This would especially be useful for data-only containers, where you might want to copy in a new configuration or private key.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jun 12, 2015

Member

@ebuchman for secrets (private keys etc.) you might want to subscribe to #13490, which is all about that.

Member

thaJeztah commented Jun 12, 2015

@ebuchman for secrets (private keys etc.) you might want to subscribe to #13490, which is all about that.

@warden

This comment has been minimized.

Show comment
Hide comment
@warden

warden Jun 14, 2015

+1 to merge and set milestone.

warden commented Jun 14, 2015

+1 to merge and set milestone.

@deathemperor

This comment has been minimized.

Show comment
Hide comment
@deathemperor

deathemperor Jul 7, 2015

+1 I need it like right now at this moment

deathemperor commented Jul 7, 2015

+1 I need it like right now at this moment

@baldwinSPC

This comment has been minimized.

Show comment
Hide comment
@baldwinSPC

baldwinSPC Jul 9, 2015

+1 This would be handy. I need it as well for a production app we're building.

baldwinSPC commented Jul 9, 2015

+1 This would be handy. I need it as well for a production app we're building.

@missedone

This comment has been minimized.

Show comment
Hide comment
@missedone

missedone Jul 10, 2015

+1 on this as it's really must have feature for me, for now as a workaround, i transfer the file via stdin of "docker exec" REST API call, but it's not convenient
please milestone this, thanks

missedone commented Jul 10, 2015

+1 on this as it's really must have feature for me, for now as a workaround, i transfer the file via stdin of "docker exec" REST API call, but it's not convenient
please milestone this, thanks

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 10, 2015

Member

The pull requests implementing this are currently on the roadmap for 1.8; see #14058 and #13171. (That's no guarantee that they will make it though)

Member

thaJeztah commented Jul 10, 2015

The pull requests implementing this are currently on the roadmap for 1.8; see #14058 and #13171. (That's no guarantee that they will make it though)

@icecrime icecrime closed this in #13171 Jul 21, 2015

@jlhawn

This comment has been minimized.

Show comment
Hide comment
@jlhawn

jlhawn Jul 22, 2015

Contributor

🎉 be sure to try out the Release Candidate for Docker 1.8 when it comes out!

Contributor

jlhawn commented Jul 22, 2015

🎉 be sure to try out the Release Candidate for Docker 1.8 when it comes out!

@duglin

This comment has been minimized.

Show comment
Hide comment
@duglin

duglin Jul 22, 2015

Contributor

Congrats Josh!!!

-Doug

Sent from my iPhone

On Jul 21, 2015, at 5:11 PM, Josh Hawn notifications@github.com wrote:

be sure to try out the Release Candidate for Docker 1.8 when it comes out!


Reply to this email directly or view it on GitHub.

Contributor

duglin commented Jul 22, 2015

Congrats Josh!!!

-Doug

Sent from my iPhone

On Jul 21, 2015, at 5:11 PM, Josh Hawn notifications@github.com wrote:

be sure to try out the Release Candidate for Docker 1.8 when it comes out!


Reply to this email directly or view it on GitHub.

@erikh

This comment has been minimized.

Show comment
Hide comment
@erikh

erikh Jul 22, 2015

Contributor

+1 congrats. Been watching this thing for a long time now. :)

-Erik

On Jul 21, 2015, at 6:09 PM, Doug Davis notifications@github.com wrote:

Congrats Josh!!!

-Doug

Sent from my iPhone

On Jul 21, 2015, at 5:11 PM, Josh Hawn notifications@github.com wrote:

be sure to try out the Release Candidate for Docker 1.8 when it comes out!


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub #5846 (comment).

Contributor

erikh commented Jul 22, 2015

+1 congrats. Been watching this thing for a long time now. :)

-Erik

On Jul 21, 2015, at 6:09 PM, Doug Davis notifications@github.com wrote:

Congrats Josh!!!

-Doug

Sent from my iPhone

On Jul 21, 2015, at 5:11 PM, Josh Hawn notifications@github.com wrote:

be sure to try out the Release Candidate for Docker 1.8 when it comes out!


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub #5846 (comment).

@sunnycmf

This comment has been minimized.

Show comment
Hide comment
@sunnycmf

sunnycmf Aug 20, 2015

👍 congrats!

sunnycmf commented Aug 20, 2015

👍 congrats!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment