[WSL2] Mounting linux path as volume not working

[verglor]

• I have tried with the latest version of my channel (Stable or Edge)
• I have uploaded Diagnostics
• Diagnostics ID: 83925FF1-D3EE-4CBF-A54A-68AC6D60F69F/20191018171529

Expected behavior

Mounting linux path (e.g. /etc/timezone) as volume should mount that path from within wsl machine.

Actual behavior

The path c:\etc\timezone (/mnt/c/etc/timezone within wsl machine) is actually mounted as volume. If it does not exist it creates it as empty folder.

Information

• Windows Version: Win 10 Pro x64 Insider Preview 18985
• Docker Desktop Version: 2.1.4.0
• Are you running inside a virtualized Windows e.g. on a cloud server or on a mac VM: no
• WSL machine: Ubuntu 18.04

Steps to reproduce the behavior

> docker context use wsl
wsl
Current context is now "wsl"
> wsl cat /etc/timezone
Europe/Bratislava
> docker run --rm -it -v /etc/timezone:/etc/timezone alpine cat /etc/timezone
cat: read error: Is a directory
> # previous command created empty folder "c:\etc\timezone"
> rmdir c:\etc\timezone
> echo FAIL > c:\etc\timezone
> cat c:\etc\timezone
FAIL
> wsl cat /mnt/c/etc/timezone
FAIL
> docker run --rm -it -v /etc/timezone:/etc/timezone alpine cat /etc/timezone
FAIL

Notes

Be able to mount linux paths from within wsl/virtual machine is essential to be able to use my containers that I am already using in linux in production, since all of them mounts at least /etc/timezone and /etc/localtime to inherit timezone configuration of the host.

[stephen-turner]

cc @simonferquel

[simonferquel]

Thanks for reporting this. It will be fixed in an upcoming release

[verglor]

Hi @simonferquel, I just tested it with 2.1.4.0 and the problem persists.

[john-landgrave]

This appears to still be an issue on 2.1.7.0 (41536). Is there a path towards resolution on this? This is kind of a deal-breaker?

[flatline-studios]

Just to add, according to:

https://www.docker.com/blog/new-docker-desktop-wsl2-backend/

Under the Initial limitations section.

It reads:

You can only mounts files backed by your distro VHD (that means you can’t bind mounts things within /tmp, /mnt, /var/run, /proc, /sys etc.). For most people, this should not be a problem, but mounting things like /var/run/docker.sock in a container won’t work initially. We are working with Microsoft on solving this issue, a future Windows Update will bring full bind mount support.

But as /etc/ is backed by the distro VHD, this is surely a bug?

Oh, and it's still broken in 2.2.2.0 (43066)

[simonferquel]

This is no more a limitation. Are you running the docker cli from your wsl distro?

[flatline-studios]

This is no more a limitation. Are you running the docker cli from your wsl distro?

This was never an issue if running docker commands inside the WSL 2 VM, I mean, why would it? But that defeats the real point of having WSL 2 with the improved Docker compatibility in the first place.

This issue is about not being able to run:

docker run --rm -it -v /etc/timezone:/etc/timezone alpine cat /etc/timezone

From a Windows command prompt, and for it to bind the /etc/timezone file from the WSL 2 environment, in to the container.

[john-landgrave]

To be clear, the issue that @flatline-studios presented is not the problem I'm having. I'm attempting to do something like:

docker run -v ~/apps/test-app:/var/www/html -name testWp -d wordpress

And it generates the container, but the contents of the /var/www/html folder in the container are not the contents of ~/apps/test-app, verified by doing a docker exec -it testWp /bin/bash and the poking around in there.

[flatline-studios]

To be clear, the issue that @flatline-studios presented is not the problem I'm having. I'm attempting to do something like:

docker run -v ~/apps/test-app:/var/www/html -name testWp -d wordpress

And it generates the container, but the contents of the /var/www/html folder in the container are not the contents of ~/apps/test-app, verified by doing a docker exec -it testWp /bin/bash and the poking around in there.

Just to clarify...

1. You're running that command in Windows command line (not in WSL itself)?
2. The ~/apps/test-app directory, that's stored within the WSL filesystem, is not being mounted in to /var/www/html, within the wordpress container?

If the answer is yes to both of those, then we're describing the same issue.

[john-landgrave]

1. I am running this command from within WSL2 (which docker returns /usr/bin/docker)
2. That is correct.

[simonferquel]

@john-landgrave
I can't reproduce on latest edge release, directory is correctly mounted and creating/modifying files from the wsl distro reflects in the container.
Can you check the latest release ?

[flatline-studios]

@simonferquel - I'm not sure if I'm being clear enough about the issue that still exists.

C:\Development
λ docker run --rm -it -v /etc/timezone:/etc/timezone alpine cat /etc/timezone
cat: read error: Is a directory

C:\Development
λ wsl -- docker run --rm -it -v /etc/timezone:/etc/timezone alpine cat /etc/timezone
Europe/London

C:\Development
Is the directory my Windows Shell is currently in.

λ
Is my command prompt symbol (using CMDer).

---

It seems that Docker for Windows isn't smart enough to look at the local mount directory /etc/timezone, realise that this is a Unix-based directory, and attempt to mount the directory from within the WSL machine.

A simple regex check ^\/[^\/].* (https://regex101.com/r/1kO0AE/1) on the host mount directory should be sufficient to do the job (granted, I haven't thought much about all the niche cases, so the regex might not suitable, as-is).

---

My workaround at the moment is to just use two alias' in the Windows Shell:

alias docker=wsl -- docker $*
alias docker-compose=wsl -- docker-compose $*

Which then produce:

C:\Development
λ alias
Use /? for help

docker-compose=wsl -- docker-compose $*
docker=wsl -- docker $*

C:\Development
λ docker run --rm -it -v /etc/timezone:/etc/timezone alpine cat /etc/timezone
Europe/London

This is a very messy hack though, and I can imagine spending an hour pulling my hair out in a couple of months time, trying to work out why the following isn't working:

C:\Development
λ type C:\tmp\test.txt
test text

C:\Development
λ docker run --rm -it -v C:\tmp:/tmp/tmp alpine cat /tmp/tmp/test.txt
docker: Error response from daemon: mkdir C:tmp: permission denied.
See 'docker run --help'.

It would be looooooovely if Docker for Windows could just work out which filesystem we wanted to mount the directory from in the first place.

[john-landgrave]

@simonferquel I'm not showing any further updates presently and here's my version info:

This still isn't working for me but is acting weird. Here's an account of the commands that I ran (all from within a WSL shell) and the output I got:

docker run --name test -v ~/apps/test:/var/www/html -e MYSQL_ROOT_PASSWORD=##### -d --rm wordpress

Where ~/apps/test is a wordpress directory on my WSL2 instance and the real password was dropped in for the #####. I also tried this without the -d and saw the following message:

WordPress not found in /var/www/html - copying now...

Which is weird because that directory, which was mounted to /var/www/html definitely has WordPress in it. So I decided to try to debug this by mounting it to a different spot: /var/test.

docker stop test
docker rm test
docker run --name test -v ~/apps/test:/var/test -e MYSQL_ROOT_PASSWORD=##### -d --rm wordpress

Here's where things get weird. When I docker exec -it test /bin/bash into the container and do an ls /var/test the contents of that directory are identical to /var/www/html from the previous trial, and still not the directory that I'm trying to mount. So, I decided to try a fresh directory that I've never mounted and just drop it into /var/app, a directory which doesn't exist by default on the WordPress container:

docker stop test
docker rm test
docker run --name test -v ~/apps/other-app:/var/app -e MYSQL_ROOT_PASSWORD=##### -d --rm wordpress

After docker exec -it test /bin/bash'ing into the container and doing an ls /var/ I see that /var/app does indeed exist (yay!). However, when I ls /var/app the directory is empty...

My next thought was that somehow it was caching volumes that were being created or something like that? So I looked at docker volume ls but I didn't see anything suspicious there.

For reference, I have also tried absolute paths on the left side of all of the -v arguments (e.g. /home/my-user/other-app).

I have also tried rebooting, factory resetting, and checking for updates after each of those things. Any ideas?

---

@flatline-studios I think that we're definitely talking about two distinct issues, and I can't necessarily determine which one the original post is talking about (unless it's a third issue with a similar enough sounding description). If I need to make a new issue (@simonferquel ?) then I'm happy to, just don't want to muddy things up too much.

[john-landgrave]

Also, my diagnostics just finished @simonferquel and here is my diagnostics ID: B0076B2C-6E30-4CE4-B2B4-069CD453F0D3/20200304144117

[simonferquel]

@flatline-studios unfortunately we can't do that: we support integration with multiple distros, and we can't decide which to mount from. Also, historically, unix paths in mount sources from Windows means "mount something from the linuxkit environment" (which makes things like docker run -v /var/run/docker.sock:/var/run/docker.sock work). So we don't want to potentially break existing workflows there.

@john-landgrave I'll have a look at the diagnostic file.

[simonferquel]

@john-landgrave it might be related to a bug I fixed recently. Can you try this private build: https://download-stage.docker.com/win/edge/43174/Docker%20Desktop%20Installer.exe ?

[john-landgrave]

@simonferquel out for lunch right now, but I'll download it give it a shot when I get back in about an hour.

[flatline-studios]

@flatline-studios unfortunately we can't do that: we support integration with multiple distros, and we can't decide which to mount from. Also, historically, unix paths in mount sources from Windows means "mount something from the linuxkit environment" (which makes things like docker run -v /var/run/docker.sock:/var/run/docker.sock work). So we don't want to potentially break existing workflows there.

@simonferquel I appreciate that, and thanks for clearing up the reasoning behind it. Much appreciated.

Trying to find a better solution than my aliases, I tried using the \\wsl$\Ubuntu-18.04\etc\timezone, but hit issue #5888.

Is there a simpler way to get around this? Like, instead of making Docker for Windows smarter, as I suggested with the regex... Wouldn't it be much easier to just tell Docker what we want. Like:

docker run --rm -it -v wsl_default//etc/timezone:/etc/timezone alpine cat /etc/timezone
                       ^^^^^^^^^^^^
docker run --rm -it -v wsl_Ubuntu-18.04//etc/timezone:/etc/timezone alpine cat /etc/timezone
                       ^^^^^^^^^^^^^^^^^

docker run --rm -it -v ?_default//etc/timezone:/etc/timezone alpine cat /etc/timezone
                       ^^^^^^^^^^
docker run --rm -it -v ?_Ubuntu-18.04//etc/timezone:/etc/timezone alpine cat /etc/timezone
                       ^^^^^^^^^^^^^^^

docker run --rm -it -v default//etc/timezone:/etc/timezone alpine cat /etc/timezone
                       ^^^^^^^^
docker run --rm -it -v Ubuntu-18.04//etc/timezone:/etc/timezone alpine cat /etc/timezone
                       ^^^^^^^^^^^^^

Or something similar?

Using // between the WSL VM distro and the directory to mount inside it seemed like a good idea. As / is an illegal character in both Windows and *nix based directories and filenames, I can't see a situation where // could appear without throwing an error under normal circumstances, so should make the ideal separator?

---

You mentioned the need to support multiple distros, and forgive my ignorance (I'm not well versed in the lower level Docker for Windows workings), but would that still be a problem with my proposed solution(s)?

[john-landgrave]

@simonferquel no dice: still have the same issue repeating the steps from my previous post.

If it helps, when I just mounted ~/apps after trying to specifically mount ~/apps/test and inspected /var/apps there was only one folder in there: test. I then Restarted docker (using the option in the menu on the tray) and executed only the mount -v /home/my_user/apps:/var/apps and the directory was empty inside the container (that directory has a lot of stuff in it, if that matters).

Should I uninstall Docker before installing the build that you sent over? It seemed to update to 2.2.3.0 fine?

[simonferquel]

Upgrading should be ok. Can you send a new diagnostic ? (This build as slightly more verbose logs)

[chadbr]

I had a similar issue today and gave up...

Running latest edge, kubernetes, wsl2 on insider 2004, wsl2 enabled

I have a deployment with specifications like this:

        volumeMounts:
        - mountPath: /kinetix-core/dist
          name: kinetix-core-dist
      volumes:
      - name: kinetix-core-dist
        hostPath:
          # directory location on host
          path:  /host_mnt/c/dev/vsts/kinetix-core/dist
          # this field is optional
          type: Directory

I received errors when k8 trying to start the image:

hostPath type check failed │: /host_mnt/c/dev/vsts/kinetix-core/dist is not a directory

---

I tried changing all the /host_mnt/c to /mnt/c like they appear in WSL with no luck...

---

Downgraded to Stable without WSL2 integration and now all is working fine.

Hope that helps...

[simonferquel]

@chadbr this is not the same issue. Mapping to windows drives is a bit different on wsl. To find which path to use, you can first create a container like docker create -v c:\something:/somewhere alpine and inspect it to see how the path was rewritten.

[john-landgrave]

@simonferquel I'm running diagnostics now and will drop the ID in here when I have it.

---

EDIT: Here you go @simonferquel: B0076B2C-6E30-4CE4-B2B4-069CD453F0D3/20200305184205

[simonferquel]

@john-landgrave
I see a wordpress running from the Windows CLI, but not from Ubuntu. Can you repro the issue and then upload a diagnostic (while the container is still running) ?
If I can't find the root issue with logs, would you be able to make a zoom call sometime ?

[john-landgrave]

@simonferquel hmmm, I guess it could be user error? (Un)fortunately I'm in the hospital right now welcoming my new son into the world, but I'll be back at it next week. I would be open to doing a Zoom call when I get back if the diagnostics and/or me trying it again doesn't work.

[simonferquel]

Congratulations!
Don't worry, I' m sure we'll be able to work this through once you're back :)

[stephen-turner]

Wow, I think that's the first time we've had a bug report from the maternity ward. Congratulations!

[john-landgrave]

Thanks all! Babies are awesome and arguably a harder problem than building software on Linux in Windows!

[john-landgrave]

Hey @simonferquel I'm back at it now (though in quarantine like the rest of the world?). Here's the diagnostic ID with the container running: B0076B2C-6E30-4CE4-B2B4-069CD453F0D3/20200326211918

[davclark]

I feel like it's still a bit unclear what we're trying to do at Gigantum:

• We run a management application that runs in Docker. It communicates with the docker daemon via a bind-mount of the docker.sock (essentially docker -v /var/run/docker.sock:/var/run/docker.sock).
• We would love to host user data on a WSL2 distro (as advised by various Docker developers), but we cannot find ANY workable way to mount WSL2
• So for now, our application has to use Windows-hosted bind-mounts

It seems that development priorities have been rapidly evolving in this general area (e.g., this comment and that we're not the only ones trying to do something like this. Our application was temporarily broken and we already deployed more complex logic that grabs the target bind-mount via inspecting the rewritten source in a test container. Maybe a bit of wasted effort given the above-linked change-of-heart. But on the flip side, it seems that something like the approach we already deployed should work here and be robust to changes - we're willing to do considerable work to be able to mount WSL2 volumes from a running container!

Should I make another ticket? Or does my use case fit here?

EDIT: I think I may have been confused about the persistence of mount -o bind style mounts. The following seems to work. Is this crazy? It works for now. If it is likely to keep working, I'd like to run with this (now that I figured out bind-mounts don't persist across "reboots" of the distro).

WLinux$ sudo mount -o bind ~/gigantum /mnt/wsl/gigantum
WLinux$ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker
/ # docker run -it --rm -v /mnt/wsl/gigantum:/mnt/gigantum ubuntu

[simonferquel]

If you run docker run -v /var/run/docker.sock from a wsl2 distro, it will bind mount the wsl proxy. So whatever bind mounts you ask for from this container will be done just like from this distro (it does not work when running from win32 though). I am pretty sure it would make your scenario much more simple

[davclark]

Just want to confirm that this does work!

I'm curious if the API has stabilized for when you call from windows also... so if I do a -v /var/run/docker.sock:/var/run/docker.sock from Windows, will the launched container (call it container 1) get an API endpoint that correctly re-writes bind-mount paths like -v c:/Users/davclark/gigantum:/mnt/gigantum from within container 1?

Or should we continue to inspect a container to get the re-written source path? (we do this currently - hopefully it doesn't break!)

[simonferquel]

Previously we did not do anything particular when mounting /var/run/docker.sock from Windows, but it has changed very recently (not sure if it is in a public build yet). We now rewrite /var/run/docker.sock to the win32 proxy socket (so windows path mount sources will just work)

[TroyLaurin]

I'm not sure if there's a way I can vote for the ability to mount wsl paths using the -v \\wsl$ syntax, but I suspect it would help me in my usecase. Specifically I would like to use Docker Desktop on WSL2 to replace minikube for local kubernetes development, but it requires being able to mount wsl hostPaths into our kubernetes containers.

[simonferquel]

We are evaluating something a bit different... what if we exposed a CLI context for each integrated WSL distro, such that you could from Windows call docker --context wsl-ubuntu run -v /home/user/source:/somewhere .... ?

[TroyLaurin]

After sleeping on it I went back to a different issue and your comment there: #5325 (comment)
On first reading I missed that your command list has a scrollbar and the actual mount command required scrolling to the right! That took me further and my questions are specifically around kubernetes rather than raw docker commands so I'll take the discussion over to that thread rather than derail this any further.

[verglor]

Hi @simonferquel, the main point of this issue was that I wanted to use the same docker-compose.yml (or complex docker command) for both linux and windows. In this regard using \\$wsl or --context is not feasible. However since you need to support multiple contexts I would suggest to use that --context switch to specify context, but without it just use the default (or current) context - I mean that when you type just wsl. Then if you preserve special handling of paths like /var/run/docker.sock it should work seamlessly in both linux and windows.

[davclark]

what if we exposed a CLI context for each integrated WSL distro, such that you could from Windows call docker --context wsl-ubuntu run -v /home/user/source:/somewhere .... ?

Sorry to be chiming in on this somewhat late - it relates to the question I was asking on Slack (#docker-desktop-win) .

Does the above thinking include the ability to use libraries to access Docker from the perspective of a WSL distro? In our case, we currently use dockerode - but a casual inspection shows me that several libraries in multiple languages only accept either a file path or an HTTP URL. It would be great to have some syntax to specify the target proxy for these libraries.

EDIT: one approach could be for my application to create a named pipe that transparently wraps the WSL2 socket?

[SJRM78]

Hello. Facing the same issue with WSL2. (i just have one WSL2 Linux Distro installed).
Is there any update on this? How can i specify docker (WSL2, from Linux shell) to munt a folder that resides inside that Linux WSL2 distro?

Thanks

[simonferquel]

@SJRM78 from a WSL 2 terminal, you can just docker run -v /my/sources:/data alpine and the /my/sources directory from your wsl distro will be correctly mounted.

[SJRM78]

Hello. This is what i am doing (all from WSL2 terminal)

docker volume create --opt type=bind --opt device=/home/allusr/volsample samplevol
docker inspect samplevol

i get this :

[
    {
        "CreatedAt": "2020-07-02T05:24:25Z",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/samplevol/_data",
        "Name": "samplevol",
        "Options": {
            "device": "/home/allusr/volsample",
            "type": "bind"
        },
        "Scope": "local"
    }
]

then i run

docker run -it -v samplevol:/home/app --rm busybox

and i get this error
docker: Error response from daemon: error while mounting volume '/var/lib/docker/volumes/samplevol/_data': failed to mount local volume: mount /home/allusr/volsample:/var/lib/docker/volumes/samplevol/_data: no such device.

However, if i run

docker run -it -v /home/allusr/volsample:/home/app --rm busybox

it works. I can find inside the docker container, in the /home/app folder the files expected.

There is a similar problem mentioned above, but with kubernetes
This is the configuration part related to volumes

    volumeMounts:
      - name: code
        mountPath: /home/app/src
  volumes:
    - name: code
      hostPath:
        path: /home/allusr/volsample/src

but the folder /home/app/src in the kubernetes pod is empty. no files there

EDIT : I have run the same commands with alpine instead of busybox, and the results are the same

[simonferquel]

Ah, indeed, named volumes of type bind are not supported. You need to use bind mounts, not named volume mounts with type bind.
I think it does not work with Hyper-V and windows source directories either.

[SJRM78]

thanks. i can do that in docker cli. :)
but, how should i do it for kubernetes ?

I'm curious, is there a plan to support named volumes ?

Thanks

[TroyLaurin]

@SJRM78 It is possible to get this working though it's not persistent - see my comment #5325 (comment) for how I got it working, but note the comment earlier in that thread: "Please note that this will break if you restart without recreating the bind mount before running docker though."

So from your example, you should do something like:

mkdir /mnt/wsl/home
sudo mount --bind /home/ /mnt/wsl/home

Then change the host path in your configuration to /run/desktop/mnt/host/wsl/home/allusr/volsample/src

[SJRM78]

Thanks @TroyLaurin i will try now.
As i don´t want to store any data there, just need to read files form it, i think it should work.

It would be great if in a future version of docker this is possible to be handled in a more transparent and friendly way

[simonferquel]

Kubernetes is much more complicated for us to work with, as we can't really intercept api calls and rewrite things, as most pod creations are done indirectly by server-side components instead of from an identifiable client.

However, what we can probably do is provide a persistent volume class driver that you could use in your kubernetes yaml (we already implement a storage driver that stores volumes data in docker-desktop-data distro).

The idea would be to declare a persistent volume (or volume claim) with StorageDriver set as com.docker.desktop/wsl-storage and with the source distro and path within this distro set as annotations, or in some form that our storage driver could intercept.

[davclark]

I just wanted to drop an update here and relay that for our desktop app for Gigantum on Windows, we are just going to do a subshell command to launch our container from within the Linux distro of choice. It's a little annoying, because you need to do some escaping to make sure Powershell doesn't replace variables, but this works from Powershell for example:

wsl -e bash -c 'docker run --rm -it -v ${HOME}:/mnt/home -v /var/run/docker.sock:/var/run/docker.sock -e HOME=${WSL_HOME} docker sh'

Probably you won't need the bash -c if you're doing a subprocess from your favorite runtime, but this is enough to get you started poking around. You can run docker successfully from that command line with path-rewriting that hits your default WSL2 distro (change the distro with the wsl -d option). So this works from inside that container:

docker run --rm -it -v ${WSL_HOME}:/mnt/home ubuntu bash

And you'll also see the dynamically determined home directory at /mnt/home in either layer of that matroishka set (e.g., the ubuntu or the docker container).

[jtsalten]

Have the same problem/need.
I want to define a named volume but decide to which path points out. It makes thinks easier defining docker-compose files.

To get it, I define the volume in advance:

docker volume create --drive local --opt type=none --opt o=bind --opt device=/home/mytest/durable myOwnVolumeBind

Then, within the docker-compose.yml I use it the volume as external:
.... volume: myOwnVolumeBind: external: true ....

Seems to work at the first execution but if I stop the containers (docker-compose down) and try to up them again, it fails trying to mount the volume... saying:
ERROR: for sharding_node2_1 Cannot start service node2: error while mounting volume '/var/lib/docker/volumes/myOwnVolumeBind/_data': failed to mount local volume: mount /run/desktop/mnt/host/wsl/docker-desktop-bind-mounts/Ubuntu-20.04/46565b0cdfb27e1c6f9311ee05188e1104e98034a9b364fa3ae2db8076f54f93:/var/lib/docker/volumes/myOwnVolumeBind/_data, flags: 0x1000: no such file or directory

Why it fails? and... what it works just the first time??

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked