New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows volume mapping works only from home directory #2548

Closed
tooda02 opened this Issue Dec 15, 2015 · 29 comments

Comments

Projects
None yet
@tooda02

tooda02 commented Dec 15, 2015

In Windows, the Getting Started example works correctly only if the working directory is under the user's home directory (C:\users\username). Otherwise, it fails with the message:

python: can't open file 'app.py': [Errno 2] No such file or directory

which occurs because the /code directory is empty in the container.

The example Dockerfile is:

FROM python:2.7
ADD . /code
WORKDIR /code
RUN pip install -r requirements.txt
CMD python app.py

and docker-compose.yml contains:

volumes:
  - .:/code

A standard Docker build produces a working container (that is, /code/app.py exists) regardless of the working directory. However, docker-compose up creates a working image only if run from C:\users\username or one of its subdirectories.

@dnephin

This comment has been minimized.

Show comment
Hide comment
@dnephin

dnephin Dec 15, 2015

Contributor

Yes, this is because Docker Machine only mounts that one shared folder by default. There's nothing we can do about this in Compose.

Contributor

dnephin commented Dec 15, 2015

Yes, this is because Docker Machine only mounts that one shared folder by default. There's nothing we can do about this in Compose.

@tooda02

This comment has been minimized.

Show comment
Hide comment
@tooda02

tooda02 Dec 15, 2015

@dnephin If I take docker-compose out of the picture by running:

docker build -t web .
docker run web

the container starts without error and I can exec a bash shell that shows the directory:

image

Note that the example above is running outside the home directory. So there is something about the interface between docker-compose and docker-machine that's causing this error.

tooda02 commented Dec 15, 2015

@dnephin If I take docker-compose out of the picture by running:

docker build -t web .
docker run web

the container starts without error and I can exec a bash shell that shows the directory:

image

Note that the example above is running outside the home directory. So there is something about the interface between docker-compose and docker-machine that's causing this error.

@dnephin

This comment has been minimized.

Show comment
Hide comment
@dnephin

dnephin Dec 15, 2015

Contributor

There is no volume in that example. If you were to use docker run -v $PWD:/code you'd have the same problem.

You can remove the volumes: ... from the compose file to get it working with compose as well.

Contributor

dnephin commented Dec 15, 2015

There is no volume in that example. If you were to use docker run -v $PWD:/code you'd have the same problem.

You can remove the volumes: ... from the compose file to get it working with compose as well.

@tooda02

This comment has been minimized.

Show comment
Hide comment
@tooda02

tooda02 Dec 15, 2015

Thanks! You are correct - removing volume makes it work. So I'd suggest that you update the Getting Started example to eliminate the Volume specification (or at the least, include a warning about this problem).

tooda02 commented Dec 15, 2015

Thanks! You are correct - removing volume makes it work. So I'd suggest that you update the Getting Started example to eliminate the Volume specification (or at the least, include a warning about this problem).

@aanand

This comment has been minimized.

Show comment
Hide comment
@aanand

aanand Dec 15, 2015

Contributor

The volume config is important, as it lets you edit the code and see changes immediately, but we don't actually demonstrate that.

There was a suggestion long ago to add the volume config in a separate step after getting the app running, and demonstrating the benefits. I still think that'd be a good change, and you're right that we should also warn about mounting issues.

Contributor

aanand commented Dec 15, 2015

The volume config is important, as it lets you edit the code and see changes immediately, but we don't actually demonstrate that.

There was a suggestion long ago to add the volume config in a separate step after getting the app running, and demonstrating the benefits. I still think that'd be a good change, and you're right that we should also warn about mounting issues.

@mikkotikkanen

This comment has been minimized.

Show comment
Hide comment
@mikkotikkanen

mikkotikkanen Feb 2, 2016

Separate step would be nice. That's what I ended up doing in a project that uses docker-compose for local dev environment, since not everyone has similar mountings but at least they can get the thing running immediately and if/when they need the live updates, they can see step n+1.

mikkotikkanen commented Feb 2, 2016

Separate step would be nice. That's what I ended up doing in a project that uses docker-compose for local dev environment, since not everyone has similar mountings but at least they can get the thing running immediately and if/when they need the live updates, they can see step n+1.

@tarach

This comment has been minimized.

Show comment
Hide comment
@tarach

tarach Feb 16, 2016

So there is no way around it? I tried mounting manually directory on other partition:

docker-machine.exe stop default
VBoxManage.exe sharedfolder add default --name "d/projects" --hostpath "D:\projects"
VBoxManage.exe showvminfo default | grep -i share -A4
docker-machine.exe start default
docker-compose up -d
// did not work
docker run -it -v "//d/projects:/d/projects" nginx /bin/sh
// did work to some extent

Hoping docker-compose will pick up on that automatically. But not only he didn't but also docker machine did mount only like 1 directory in projects ( the directory with docker files and also not all of them )

There isn't a way to make it work?

tarach commented Feb 16, 2016

So there is no way around it? I tried mounting manually directory on other partition:

docker-machine.exe stop default
VBoxManage.exe sharedfolder add default --name "d/projects" --hostpath "D:\projects"
VBoxManage.exe showvminfo default | grep -i share -A4
docker-machine.exe start default
docker-compose up -d
// did not work
docker run -it -v "//d/projects:/d/projects" nginx /bin/sh
// did work to some extent

Hoping docker-compose will pick up on that automatically. But not only he didn't but also docker machine did mount only like 1 directory in projects ( the directory with docker files and also not all of them )

There isn't a way to make it work?

@jennet

This comment has been minimized.

Show comment
Hide comment
@jennet

jennet Jul 7, 2016

have you shared the drive containing the project directory in the docker settings? (see shared drive section here)

Once I'd done that I was able to run the Getting Started tutorial fine.

jennet commented Jul 7, 2016

have you shared the drive containing the project directory in the docker settings? (see shared drive section here)

Once I'd done that I was able to run the Getting Started tutorial fine.

@vweevers

This comment has been minimized.

Show comment
Hide comment
@vweevers

vweevers Jul 13, 2016

Here's a workaround for volumes outside of C:\Users. I'm on Win8.1, Docker Machine 0.7.0, no git bash. Add a shared folder and mount it inside the VM:

cd d:\my-project
vboxmanage sharedfolder add default --name my-project --hostpath "%cd%" --transient

docker-machine ssh default "sudo mkdir -p /d/my-project"
docker-machine ssh default "sudo mount -t vboxsf -o defaults,uid=`id -u docker`,gid=`id -g docker` my-project /d/my-project"

docker-compose up

Notes:

  • Add --readonly to the vboxmanage sharedfolder command to create a read-only share
  • If vboxmanage is not in your PATH, try "%VBOX_MSI_INSTALL_PATH%\vboxmanage"
  • Replace default with %DOCKER_MACHINE_NAME% if necessary
  • You have to do these steps for every project

vweevers commented Jul 13, 2016

Here's a workaround for volumes outside of C:\Users. I'm on Win8.1, Docker Machine 0.7.0, no git bash. Add a shared folder and mount it inside the VM:

cd d:\my-project
vboxmanage sharedfolder add default --name my-project --hostpath "%cd%" --transient

docker-machine ssh default "sudo mkdir -p /d/my-project"
docker-machine ssh default "sudo mount -t vboxsf -o defaults,uid=`id -u docker`,gid=`id -g docker` my-project /d/my-project"

docker-compose up

Notes:

  • Add --readonly to the vboxmanage sharedfolder command to create a read-only share
  • If vboxmanage is not in your PATH, try "%VBOX_MSI_INSTALL_PATH%\vboxmanage"
  • Replace default with %DOCKER_MACHINE_NAME% if necessary
  • You have to do these steps for every project
@aanand

This comment has been minimized.

Show comment
Hide comment
@aanand

aanand Jul 13, 2016

Contributor

You might want to give the Docker for Windows beta a try - it uses a different approach for file syncing and doesn't have the C:\Users restriction.

Contributor

aanand commented Jul 13, 2016

You might want to give the Docker for Windows beta a try - it uses a different approach for file syncing and doesn't have the C:\Users restriction.

@vweevers

This comment has been minimized.

Show comment
Hide comment
@vweevers

vweevers Jul 15, 2016

Made a tool for it: docker-share. Give it a spin, when Docker for Windows is not an option for you. In its simplest form, the workflow is:

docker-machine start
@FOR /f "tokens=*" %i IN ('docker-machine env') DO @%i

cd my-project
docker-share mount -t
docker-compose up

vweevers commented Jul 15, 2016

Made a tool for it: docker-share. Give it a spin, when Docker for Windows is not an option for you. In its simplest form, the workflow is:

docker-machine start
@FOR /f "tokens=*" %i IN ('docker-machine env') DO @%i

cd my-project
docker-share mount -t
docker-compose up
@wulftone

This comment has been minimized.

Show comment
Hide comment
@wulftone

wulftone Jul 17, 2016

I'm also having this problem in Windows 10. I'm running Docker for Windows (latest, 1.12.0-rc4-beta19) to no avail. I can ADD files but I cannot mount volumes. My project is in a subdirectory under C:\Users\me\Whatever\DockerProject. Would love to see this resolved!

wulftone commented Jul 17, 2016

I'm also having this problem in Windows 10. I'm running Docker for Windows (latest, 1.12.0-rc4-beta19) to no avail. I can ADD files but I cannot mount volumes. My project is in a subdirectory under C:\Users\me\Whatever\DockerProject. Would love to see this resolved!

@klawhk

This comment has been minimized.

Show comment
Hide comment
@klawhk

klawhk Aug 2, 2016

@wulftone I am using Windows 10 too. I executed "docker inspect" and found the volume was mounted but the error still came out.
Finally I resolved by going to settings and set set Shared Drives. I hope it can help!
image

klawhk commented Aug 2, 2016

@wulftone I am using Windows 10 too. I executed "docker inspect" and found the volume was mounted but the error still came out.
Finally I resolved by going to settings and set set Shared Drives. I hope it can help!
image

@philicious

This comment has been minimized.

Show comment
Hide comment
@philicious

philicious Aug 4, 2016

As I also still see these problems with devs using Windows, I came over https://forums.docker.com/t/volume-mounts-in-windows-does-not-work/10693/105 and it seems this still is a problem and only working for some users, for others it does work after fiddling around with firewall, security policies etc and for others it just doesnt work

philicious commented Aug 4, 2016

As I also still see these problems with devs using Windows, I came over https://forums.docker.com/t/volume-mounts-in-windows-does-not-work/10693/105 and it seems this still is a problem and only working for some users, for others it does work after fiddling around with firewall, security policies etc and for others it just doesnt work

@enjikaka

This comment has been minimized.

Show comment
Hide comment
@enjikaka

enjikaka Aug 16, 2016

@klawhk Thank you. That solved it for me. :)

enjikaka commented Aug 16, 2016

@klawhk Thank you. That solved it for me. :)

@philicious

This comment has been minimized.

Show comment
Hide comment
@philicious

philicious Aug 16, 2016

oh a little update: for me the problems were because of a firewall integrated in AV (AntiVirus), so shut that OFF. Windows Firewall itself doesnt seem to cause problems tho.

philicious commented Aug 16, 2016

oh a little update: for me the problems were because of a firewall integrated in AV (AntiVirus), so shut that OFF. Windows Firewall itself doesnt seem to cause problems tho.

@barrystaes

This comment has been minimized.

Show comment
Hide comment
@barrystaes

barrystaes Oct 19, 2016

The above workaround comment by @vweevers works great. When in doubt use docker inspect to find the mounted folders. I just mounted the entire project folder and that works fine.

barrystaes commented Oct 19, 2016

The above workaround comment by @vweevers works great. When in doubt use docker inspect to find the mounted folders. I just mounted the entire project folder and that works fine.

@ztwo79

This comment has been minimized.

Show comment
Hide comment
@ztwo79

ztwo79 Oct 20, 2016

thank you @vweevers , great tool.

ztwo79 commented Oct 20, 2016

thank you @vweevers , great tool.

@markl17

This comment has been minimized.

Show comment
Hide comment
@markl17

markl17 Feb 15, 2017

by me it still automounts c:\users even though it is says in shared drives c how does one convince it to share c:\

markl17 commented Feb 15, 2017

by me it still automounts c:\users even though it is says in shared drives c how does one convince it to share c:\

@Lampei

This comment has been minimized.

Show comment
Hide comment
@Lampei

Lampei Jun 11, 2017

I think this issue could be much more simply resolved by some more error handling in the docker-compose up command. If there was a way to inform the user (I guess info vs error) that the volume mount in windows using docker toolbox did not work because folder xyz does not exist (and then display where docker-compose is looking) would solve 99% of the problem. I know that much of my time was spent thinking it was something with the container I was using rather than not using my c:\Users\ directory even after I read the documentation. If I had seen in the console log that the mounting was from c:\Users I probably would have worked forward from there...or even to update the shared folders in VirtualBox.

Lampei commented Jun 11, 2017

I think this issue could be much more simply resolved by some more error handling in the docker-compose up command. If there was a way to inform the user (I guess info vs error) that the volume mount in windows using docker toolbox did not work because folder xyz does not exist (and then display where docker-compose is looking) would solve 99% of the problem. I know that much of my time was spent thinking it was something with the container I was using rather than not using my c:\Users\ directory even after I read the documentation. If I had seen in the console log that the mounting was from c:\Users I probably would have worked forward from there...or even to update the shared folders in VirtualBox.

@rafaelsoaresbr

This comment has been minimized.

Show comment
Hide comment
@rafaelsoaresbr

rafaelsoaresbr Jun 16, 2017

Anyway to share another drive using Docker Toolbox/Kitematic?

rafaelsoaresbr commented Jun 16, 2017

Anyway to share another drive using Docker Toolbox/Kitematic?

@markl17

This comment has been minimized.

Show comment
Hide comment
@markl17

markl17 Jun 16, 2017

In Windows to mount another drive is easy but to get c: to mount is impossible it only mounts c:\users`name of the user`

markl17 commented Jun 16, 2017

In Windows to mount another drive is easy but to get c: to mount is impossible it only mounts c:\users`name of the user`

@Lampei

This comment has been minimized.

Show comment
Hide comment
@Lampei

Lampei Jun 16, 2017

Well, it's not impossible @markl17 :) The way to do it via VirtualBox (as this is the way I handle it on my machine), is to stop your running VM if it's running, click on the "Shared Drive" tab, and then add a new link to your C:\ drive. Now start up your VM again. Then from here you have full access to your entire c drive rather than just sub folders in your C:\Users folder. It even gets auto-named for you to C_DRIVE Then you can use this in your yml file and add your volumes.

Lampei commented Jun 16, 2017

Well, it's not impossible @markl17 :) The way to do it via VirtualBox (as this is the way I handle it on my machine), is to stop your running VM if it's running, click on the "Shared Drive" tab, and then add a new link to your C:\ drive. Now start up your VM again. Then from here you have full access to your entire c drive rather than just sub folders in your C:\Users folder. It even gets auto-named for you to C_DRIVE Then you can use this in your yml file and add your volumes.

@marcospgp

This comment has been minimized.

Show comment
Hide comment
@marcospgp

marcospgp Jul 18, 2017

@Lampei exactly, I spent hours debugging my docker files because of this. Since this was my first time using docker, I was pretty lost as to what I was doing wrong.

I finally tried getting a boilerplate project running out of the box and realized it wasn't my fault when it didn't work.

In an obscure corner of the internet, I found something about C:/users and tried moving the project folder to my desktop. Worked like a charm.

Why is this problem so hidden? It would be so easy to have some error message when you tried to mount a path that was not found.

Where did I go wrong in this? What resource should I be paying attention to that I missed, in order to be aware of this and avoid spending hours solving a nonexistent problem in my code?

marcospgp commented Jul 18, 2017

@Lampei exactly, I spent hours debugging my docker files because of this. Since this was my first time using docker, I was pretty lost as to what I was doing wrong.

I finally tried getting a boilerplate project running out of the box and realized it wasn't my fault when it didn't work.

In an obscure corner of the internet, I found something about C:/users and tried moving the project folder to my desktop. Worked like a charm.

Why is this problem so hidden? It would be so easy to have some error message when you tried to mount a path that was not found.

Where did I go wrong in this? What resource should I be paying attention to that I missed, in order to be aware of this and avoid spending hours solving a nonexistent problem in my code?

@dotview

This comment has been minimized.

Show comment
Hide comment
@dotview

dotview Jan 16, 2018

Perfect! Thank you @vweevers !

dotview commented Jan 16, 2018

Perfect! Thank you @vweevers !

@shin- shin- closed this Jan 25, 2018

@tomahim

This comment has been minimized.

Show comment
Hide comment
@tomahim

tomahim Aug 30, 2018

Just did this small bash script if it can help :

#!/bin/bash
PROJECT_PATH=${PWD}
PROJECT_NAME=${PWD##*/}

sharedfolder_exists=`vboxmanage showvminfo default | grep "Name: '${PROJECT_NAME}'"`
if [[ -z $sharedfolder_exists ]]
then
    vboxmanage sharedfolder add default --name ${PROJECT_NAME} --hostpath ${PROJECT_PATH} --transient
    docker-machine ssh default "sudo mkdir -p ${PROJECT_PATH}"
    docker-machine ssh default "sudo mount -t vboxsf -o defaults,uid=`id -u`,gid=`id -g` ${PROJECT_NAME} ${PROJECT_PATH}"
else
    echo "INFO : Shared folder already set"
fi
docker-compose up

It requires cd in the project directory (where the docker-compose.yml file is) before executing this script.

tomahim commented Aug 30, 2018

Just did this small bash script if it can help :

#!/bin/bash
PROJECT_PATH=${PWD}
PROJECT_NAME=${PWD##*/}

sharedfolder_exists=`vboxmanage showvminfo default | grep "Name: '${PROJECT_NAME}'"`
if [[ -z $sharedfolder_exists ]]
then
    vboxmanage sharedfolder add default --name ${PROJECT_NAME} --hostpath ${PROJECT_PATH} --transient
    docker-machine ssh default "sudo mkdir -p ${PROJECT_PATH}"
    docker-machine ssh default "sudo mount -t vboxsf -o defaults,uid=`id -u`,gid=`id -g` ${PROJECT_NAME} ${PROJECT_PATH}"
else
    echo "INFO : Shared folder already set"
fi
docker-compose up

It requires cd in the project directory (where the docker-compose.yml file is) before executing this script.

@marcospgp

This comment has been minimized.

Show comment
Hide comment
@marcospgp

marcospgp Oct 7, 2018

I can't believe I posted here a year ago. Hello past me! It just took me another day to rediscover the problem you solved a year ago!

Why can't we just add an error message to docker compose or docker toolbox? Please?

@shin- Can you please reopen?

marcospgp commented Oct 7, 2018

I can't believe I posted here a year ago. Hello past me! It just took me another day to rediscover the problem you solved a year ago!

Why can't we just add an error message to docker compose or docker toolbox? Please?

@shin- Can you please reopen?

@shin-

This comment has been minimized.

Show comment
Hide comment
@shin-

shin- Oct 8, 2018

Member

There's no way for Compose to know which folders you've mounted, and it is the intended behavior for non-existing folders to be created by Docker, as highlighted here: https://docs.docker.com/storage/bind-mounts/#differences-between--v-and---mount-behavior . Many users rely on that behavior persisting in future versions.

If you want to see an error when you mount an unknown folder, use the long syntax (mount syntax) instead.

Member

shin- commented Oct 8, 2018

There's no way for Compose to know which folders you've mounted, and it is the intended behavior for non-existing folders to be created by Docker, as highlighted here: https://docs.docker.com/storage/bind-mounts/#differences-between--v-and---mount-behavior . Many users rely on that behavior persisting in future versions.

If you want to see an error when you mount an unknown folder, use the long syntax (mount syntax) instead.

@marcospgp

This comment has been minimized.

Show comment
Hide comment
@marcospgp

marcospgp Oct 8, 2018

@shin- I believe even a small "Folder "path/to/folder" not found, will be created by Docker" would help.

Doesn't compose read the docker-compose.yml file? What do you mean there is no way for it to know which folders you've mounted?

Thanks for the quick feedback!!

marcospgp commented Oct 8, 2018

@shin- I believe even a small "Folder "path/to/folder" not found, will be created by Docker" would help.

Doesn't compose read the docker-compose.yml file? What do you mean there is no way for it to know which folders you've mounted?

Thanks for the quick feedback!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment