Kubernetes Volumes not correctly mounted with WSL2

[Gsyltc]

I have migrated to WSL2 - Windows 10.
Since I have the following issue :

Hostpath volume are not mounted into containers. (directory are empty)

The volumes are well created, the desired path is correct. Like

/volumes/my-cluster/services1/www

Directory "/volumes" have 777 permissions

Volumes looks like that :

vol-www 30Mi RWO Retain Bound jeedom/pvc-www hostpath 19m

PersistenVolumeClaim are bounds to the volumes

pvc-www Bound vol-www 30Mi RWO hostpath 19m

In WSL1, at the start of the deployment (or helm charts installation). If Directories does not exists, they are created. Volume are mounted to the containers and work well
Conditions : volumes must be mounted in /c/.... (not in /mnt/c/...)

With WSL2, there is no need to mount volume in /c/...
docker run -v /volumes/my-cluster/services1/www:/var/html/www my-image work well.

With kubernetes, local directories are not created. directory in container are empty. When creating a files on wsl path, it's not appears in the container, and the other way around doesn't work either.

More, the method who works with WSL1 don't work with WSL2 (mount volume in /c/...)

Thanks

Information

• Windows Version: 10 build 19037

• Docker Desktop Version: 2.1.7.0 (41536) Edge

• Kubetctl Version Client : 1.17.0
  Server : 1.15.5

• helm V3.0.1+g7c22ef9

• Docker 19.03.5

Deployment File

# Source: charts/supervision/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: supervision
  namespace: j*****m
  labels:
    app.kubernetes.io/name: supervision
    helm.sh/chart: supervision-2.0.1-beta
    app.kubernetes.io/version: "1.0"
    app.kubernetes.io/managed-by: Helm
spec: 
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: supervision
  strategy:
    rollingUpdate: 
      maxSurge: 1
      maxUnavailable : 1 
    type: RollingUpdate 
    
  # Template
  template:
    metadata:
      labels:
        app.kubernetes.io/name: supervision
    spec:
      
      hostname: supervision
      containers:
         # Image
       ....
       - image: "*****1.3"
          imagePullPolicy: IfNotPresent
          # Volumes
          volumeMounts:
            - name: logs
              mountPath: /var/log/mosquitto
       ....
      # Volumes
      volumes:
      - name: logs
        persistentVolumeClaim:
          claimName: "pvc-supervision-logs"
      - name: persistence
        persistentVolumeClaim:
          claimName: "pvc-supervision-persistence"

---

Source: charts/supervision/templates/persistent-volumes.yaml

kind: PersistentVolume
apiVersion: v1
metadata: 
  name: "vol-persistence"
  labels:
    volume: persistence
spec:
  storageClassName: hostpath
  capacity:
    storage: 30Mi
  accessModes:
  - ReadWriteOnce
  hostPath:
    path: /c/Workspace/volumes/mqtt/persistence
---
# Source: charts/supervision/templates/persistent-volumes.yaml
kind: PersistentVolume
apiVersion: v1
metadata: 
  name: "vol-logs"
  labels:
    volume: logs
spec:
  storageClassName: hostpath
  capacity:
    storage: 50Mi
  accessModes:
  - ReadWriteOnce
  hostPath:
    path: /c/Workspace/volumes/mqtt/logs

---

Source: charts/supervision/templates/persistent-volume-claims.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: "pvc-supervision-persistence"
spec:
  storageClassName: hostpath
  volumeName: "vol-persistence"
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 30Mi
---
# Source: charts/supervision/templates/persistent-volume-claims.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: "pvc-supervision-logs"
spec:
  storageClassName: hostpath
  volumeName: "vol-logs"
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 50Mi

[wiruzman]

Any update on this?

[simonferquel]

Unfortunately, we don't support hostpath volumes in wsl2. (we don't support them officially with hyper-v either, but there are some potential tricks to make them work).

I'll try to find a workaround for you, but it is a tricky subject.

[wiruzman]

Unfortunately, we don't support hostpath volumes in wsl2. (we don't support them officially with hyper-v either, but there are some potential tricks to make them work).

I'll try to find a workaround for you, but it is a tricky subject.

@simonferquel I'll appreciate if you can provide the workaround for this.

[simonferquel]

So the idea here would be to leverage cross distro mounts.
Cross distro mounts is actually a tmpfs volume that is mounted in "shared" propagation mode in every distro.

You can put things there to share with other distros, and to docker daemon (that is how we handle docker bind mounts under the hood).

So, start by creating a mountpoint in there, and mount the directory you want to share with your pods:

mkdir /mnt/wsl/sample-hostmount     
mkdir somedir                                                                                                                                                                           echo hello > somedir/hello                                                                                                                                                              sudo mount --bind somedir /mnt/wsl/sample-hostmount 

/mnt/wsl is actually the mount point for the cross-distro mounts tmpfs
Docker Daemon mounts it in its /run/desktop/mnt/host/wsl directory

So from windows side (not wsl) you can try running:
docker.exe run --rm -it -v /run/desktop/mnt/host/wsl/sample-hostmount:/sample-hostmount alpine cat /sample-hostmount/hello and see the content of the file (you need to run this from windows, as from wsl, it would rewrite your bind source to expose your own distro's /run/desktop/mnt/host/wsl/sample-hostmount)

Then you can use hostmounts with paths rewritten with the /run/desktop/mnt/host/wsl prefix in your pods.

Please note that this will break if you restart without recreating the bind mount before running docker though.

[GLStephen]

@simonferquel Can you point me at some docs regarding recommended method to mount volumes in WSL2 and Docker Desktop? I have all the requisite versions, but there doesn't seem to be good docs on the recommended or preferred method to access files in the Windows environment either via mounts in WSL2 or otherwise.

[GLStephen]

To clarify it feels like there needs to be some disambiguation between the wider kubernetes docs and what is available in the WSL2/Docker Desktop install. Some task based docs would be amazing.

[simonferquel]

There is no official "mount from a wsl distro" volume support yet (if it becomes a popular demand we will change that, likely with some docker desktop specific volume plugin).
However default volume provider works out of the box (not sure if it stores files on the docker-desktop-data distro or on the windows file system though, I need to check)

[GLStephen]

@simonferquel thanks for the response, it is very possible that I'm misunderstanding some nomenclature. After some teeth nashing I got this working:

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-pv
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /run/desktop/mnt/host/c/Users/sjohnston/Documents/Development/helloworld/app
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - docker-desktop
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dir
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

How does that differ from what you said wasn't possible?

[Gsyltc]

@simonferquel thanks for the response, it is very possible that I'm misunderstanding some nomenclature. After some teeth nashing I got this working:

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-pv
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /run/desktop/mnt/host/c/Users/sjohnston/Documents/Development/helloworld/app
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - docker-desktop
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dir
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

How does that differ from what you said wasn't possible?

Your storage class is local-storage, not hostpath. ;)

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[jloic]

So the idea here would be to leverage cross distro mounts.
Cross distro mounts is actually a tmpfs volume that is mounted in "shared" propagation mode in every distro.

You can put things there to share with other distros, and to docker daemon (that is how we handle docker bind mounts under the hood).

So, start by creating a mountpoint in there, and mount the directory you want to share with your pods:

mkdir /mnt/wsl/sample-hostmount     
mkdir somedir                                                                                                                                                                           echo hello > somedir/hello                                                                                                                                                              sudo mount --bind somedir /mnt/wsl/sample-hostmount 

/mnt/wsl is actually the mount point for the cross-distro mounts tmpfs
Docker Daemon mounts it in its /run/desktop/mnt/host/wsl directory

So from windows side (not wsl) you can try running:
docker.exe run --rm -it -v /run/desktop/mnt/host/wsl/sample-hostmount:/sample-hostmount alpine cat /sample-hostmount/hello and see the content of the file (you need to run this from windows, as from wsl, it would rewrite your bind source to expose your own distro's /run/desktop/mnt/host/wsl/sample-hostmount)

Then you can use hostmounts with paths rewritten with the /run/desktop/mnt/host/wsl prefix in your pods.

Please note that this will break if you restart without recreating the bind mount before running docker though.

This should be documented somewhere (other than here I mean). Thanks a lot.

[TroyLaurin]

Definitely agree with @jloic that this needs to be easier to find. @simonferquel 's suggestion worked for me (after I discovered the scrollbar!) with these commands:

mkdir /mnt/wsl/work
sudo mount --bind /home/troy/work /mnt/wsl/work

Then the relevant parts of my pod configuration:

 Containers:
   postgres:
     Mounts:
       /docker-entrypoint-initdb.d from db-init (rw)
 Volumes:
   db-init:
     Type:          HostPath (bare host directory volume)
     Path:          /run/desktop/mnt/host/wsl/work/dev_init/20200422
     HostPathType:  Directory

We already have path translating between host and container so this is a workable step in a solution for us. Making sure the bind-mount is set up before starting the docker daemon will be a nuisance but manageable. It would be useful to know if/when there's a canonical/correct way to do this that automatically sets up the bind as required.

[simonferquel]

The challenge we have with hostmount, is that when a Pod is created, we can't know from which distro it comes (as usually PODs, are created trough deployments, statefulsets, daemonsets etc., that do not propagate origin owner identity to child resources).

[TroyLaurin]

That does make for quite a problem. I get confused reconciling host/VM/container paths and I have the complete context 😁 I'll see if we can manage with simple bind mounts like this, if we can script it or otherwise make it as simple as possible it may suffice for development.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked