-
Notifications
You must be signed in to change notification settings - Fork 18.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker Volumes not respect container folder contents #4361
Comments
If you mean that you're mounting a volume over an existing directory, and expecting to see the files from the existing directory… you wont. This is consistent with the general behavior of mount:
|
This behavior is bad. It would be nice if there was an option for this, because if a container stops turning, I'll lose my data. If I could centralize the files on the host and only let the software in the container would be ideal. An example would be having a MySQL database and write files on the host, but if I run a MySQL container when starting a volume'll lose the files and settings from default tables in MySQL |
There can't be an option for this — Docker volumes are subject to the mechanics of unix filesystem mounting. But you can certainly work around it. Some useful links:
I'd suggest you close this issue, and use public forums / IRC etc for further help. |
Thanks pda! I'll check the links these But Docker could not simply copy the contents of the container folder to put the host before creating the volume? This would greatly simplify the workflow. |
or perhaps warn that its about to mount into a non-empty directory. Its quite fun to watch when someone succeeds in mmm, Actually, a warning will help, but I think I also need to add some more documentation to that |
I think:
|
I understand that volumes are treated as mount, but I believe many people need this as a new feature because it would greatly simplify and make possible to have an environment already configured rather than run a container with volumes and only then configure it. |
Wait so is this not true? From Docker docs.
|
This issue was from Docker 0.8 days. The information in it may or may not have been true at the time, but I wouldn't pay much attention to it now, either way. |
@Paislee , there is a note further down specific to mounting host directories: https://docs.docker.com/userguide/dockervolumes/#mount-a-host-directory-as-a-data-volume
|
As paislee is pointing out, the documentation (https://docs.docker.com/engine/userguide/dockervolumes/) is wrong:
Instead the last sentence should read:
The quote by @jelazos7 seems to have been removed. /b3 |
@beetree the content isn't removed from the container though, it is "masked", because the mounted directory is mounted on top of the existing files. The files are still in the container, only not reachable |
@thaJeztah I might miss the subtle difference between "removed/replaced" and "masked". If the files can't be seen, read or written to, aren't they practically non-existent/deleted? Do the files appear if you delete the mounted directory from within the container, or do they appear if you unmount the volume(s) from the (running?) container? I understand the technical difference in the layering/storage, but to a user "removed/replaced" seems identical to "masked". |
No they're not deleted. The volume is mounted "over it", but the files in the container are untouched (not deleted). For example, take this Dockerfile; FROM ubuntu:latest
RUN mkdir -p /test/
RUN echo hello > /test/hello Build the Dockerfile, and tag the image "example"; root@dockr:~/projects/masked# docker build -t example .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM ubuntu:latest
---> 6d4946999d4f
Step 2 : RUN mkdir -p /test/
---> Using cache
---> f260463d3794
Step 3 : RUN echo hello > /test/hello
---> Running in 8f863e8e2f59
---> cef8a1760237
Removing intermediate container 8f863e8e2f59
Successfully built cef8a1760237 Run the container, using the current directory as a bind-mounted volume.
root@dockr:~/projects/masked# docker run -it --rm --privileged -v $(pwd):/test/ example Inside the root@362dc1ce612b:/# ls /test
Dockerfile However, un-mounting the volume reveals the content that is still in the container (only not accessible, because the volume is mounted over it) root@362dc1ce612b:/# umount /test
root@362dc1ce612b:/# ls /test
hello As you can see, the "hello" file is still there. |
Ah, that makes a lot of sense. Thanks @thaJeztah for explaining this! Appreciate it! |
@beetree you're welcome! |
Is there a way to run commands after the volume is mounted? I want to mount the folder that has my |
@yefim If you use an Entrypoint script it will run at container startup after the volumes have been mounted. |
So I had an existing postgresql db container that I started with the following command: I was messing around with networking and started another container with the following command: The 2nd container failed to start, and now when I try to start the first container via "docker start postgresql-redmine", this fails to start as well. Did I overwrite the first container's volume or just mount over it? Any idea how I can recover the volume and restart the container? Thanks for any assistance... |
@tcollavo you started both containers with the same bind-mounted host directory; this resulted in two PostgreSQL servers working / writing to the same directory. I can't tell if this resulted in data being overwritten by the second PostgreSQL server, permissions being changed, or something else. |
I am also running a jenkins container using docker-compose. Every time when I reboot my machine, my container gets restarted when host comes back up again, though the host directory where I have mounted /var/jenkins_home of container, gets recreated from scratch and I lose all my data. I thought, as per this thread, that this issue was with earlier version of docker-compose whereas I am running version 1.9 of docker-compose. Here are further details -
|
What is missing here is simple example how one can use host directory over non-empty I understand mechanics behind linux
And run it without passing any volume params e.g. I went through bazillion closed issues, and it looks like another one should be opened, because over and over and over again, people are asking for this, and so far all of them are closed and not even a workaround is provided. I'm not really sure what's so hard to do with allowing additional param that will copy contents of guest directory to the host directory before mounting, the same way it's copied to the volume before it's mounted. Or at least, what's the workaround to copy from image to host and then bind mount? |
I would also like to know if anyone has a good work-around for "copy from image to host and THEN bind mount" for volumes. It's extremely useful for local development with containers. |
I will show the inconsistent behavior when you mount your filesystem comparing to mounting a volume. I will do the experiment using image library/ubuntu. And I have created a folder in /tmp which I will use as the mount folder $ ls /tmp/myfolder
myfile myfile2 mount nothing:It will show the content of ubuntu's /usr/ --- some folders, which is fine. $ docker run -it ubuntu ls /usr/
bin games include lib local sbin share src mount my folder in local FSAnd when I mount myfolder to the container, the origin content of $ docker run -v /tmp/myfolder/:/usr/ -it ubuntu ls /usr/
myfile myfile2 mount a volumnHowever with the Docker Volumn the behavior is totally different from mounting local FS. It's more like mounting nothing. $ docker run -v foo:/usr/ -it ubuntu ls /usr/
bin games include lib local sbin share src So that means with the identical syntax
|
@kehao95 I believe this has to do with the following (taken from docker docs):
So what is happening here is that the contents of the This behavior has also been discussed in #18670 |
Correct; even thought the "mechanics" behind bind mounted host directories and volumes are similar, they have a different purpose, and follow different semantics: When using a (named) volume, you tell docker to create a new storage-space (if it doesn't exist) for a container to persist files outside of the container's filesystem. If that volume is empty, files from the container are copied to the volume's storage location, and the volume is mounted inside the container. Mounting a volume in the container "masks" the file that were already in that location, so what you see inside the container is the files that are present in the volume. When bind mounting a host directory you give the container permission to access a given path on the host, and any content in that path. If that directory happens to be empty, you give the container access to an empty directory. Access to the directory can be |
So there is still no workaround for this one ? 2021 already |
@turbo8p an empty named volume works to expose file from the container to the host, doesn't it? As @thaJeztah explained (emphasis mine)
That's an easy and good solution. Unless you specifically need a bind mount instead of a named volume, but I see no advantage in that. As the Dockers Volumes docs say (emphasis mine):
|
There was (https://dev.to/adamkdean/mount-docker-volume-locally-59mg) but this stopped working, annoyingly. |
@adamkdean perhaps this helps; #19990 (comment) ("named volumes with a custom storage location"), and the comments linked from there. TL;DR it's possible to create a named volume that bind-mounts files from a custom location (but it's taking advantage to |
@thaJeztah thanks Sebastiaan, that's an interesting thread! For my needs (replacing what previously worked through mapping a bind mount to named volume) I just went with a bind mount and some symbolic links for build time content. It worked well enough. Just shows there's always more than one solution! :) |
so now it's 2024 and we still have no solution |
@voss01 solution to what exactly? |
When i use a volume, the contents of my folder in container are deleted.
Is there any configuration to be given a merge the files? It is a bug?
Ubuntu version: 13.10 (Saucy)
Docker version: Docker version 0.8
VirtualBox version: 4.3.6
Vagrant version: 1.4.3
The text was updated successfully, but these errors were encountered: