subPath in volumeMount is not writable for non-root users

[thriqon]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): no

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): subpath, emptyDir, mode, permissions

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:52:34Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Cloud provider or hardware configuration: Bare metal, SSD, but also VMs
• OS (e.g. from /etc/os-release): Debian GNU/Linux 8 (jessie)
• Kernel (e.g. uname -a): 4.8.0-0.bpo.2-amd64
• Install tools:
• Others: Docker v1.12.6

What happened: When I mount an emptyDir volume using a subpath, I'm unable to write into it if my process is not running as root. When I leave out the subpath, I can.

What you expected to happen: I can write to the directory in both cases.

How to reproduce it (as minimally and precisely as possible):

kind: Pod
apiVersion: v1
metadata: { name: "test-modes" }
spec:
  volumes:
    - { name: "direct",  emptyDir: {}}
    - { name: "subpath", emptyDir: {}}
  securityContext:
    fsGroup: 100
    # games(35) is member in group users(100)
    runAsUser: 35
  containers:
    - image: alpine:latest
      name: show
      command: ["/bin/sh", "-c", "id && ls -dl /mnts/*"]
      volumeMounts:
        - name: "direct"
          mountPath: "/mnts/direct"
        - name: "subpath"
          mountPath: "/mnts/subpath"
          subPath: a/b/c

Output is:

uid=35(games) gid=35(games) groups=100(users)
drwxrwsrwx    2 root     users         4096 Feb 17 12:38 /mnts/direct
drwxr-sr-x    2 root     users         4096 Feb 17 12:38 /mnts/subpath

Anything else we need to know:

This issue is related to #39474. I know this is caused by the Docker daemon creating those directories on a bind mount. Nevertheless, I expect Kubernetes to abstract this away, since this behavior is neither documented nor IMHO intuitive.

[calebamiles]

Could someone from @kubernetes/sig-node-bugs please take a look? Thanks!

cc: @kubernetes/sig-storage-bugs

[smarterclayton]

@pmorie @kubernetes/sig-storage-misc

[dchen1107]

cc/ @saad-ali

[wongma7]

attempted fix created here #43775

[wongma7]

I created a cherrypick PR for 1.6 here: #44782 since this bug is against 1.6. Could someone (who agrees this could be cherrypicked) please add the cherrypick-candidate label + milestone to the original PR #43775, thank you!

edit: sorry, it's against 1.5. Should it be cherrypicked back to that as well?

[rakeshvanga]

I'm still facing this issue with Kubernetes version 1.12.8 on a AKS cluster when I mount a Azure Storage as azureFile share.
Did anyone know if the issue is fixed for all types of mounts and not only just the emptyDir?

[alegmal]

same issue on GCP @ k8s 1.14

[ronaldpetty]

k8s 1.17 via eks as well

[Saksow]

So what's the recommendation? Not using subPath?