There seems to be no way of setting permissions on the mount path #81676

jeffsaremi · 2019-08-20T18:44:21Z

I have posted this to StackOverflow (https://stackoverflow.com/questions/57566773/how-to-change-the-umask-when-mounting-volumes-in-kubernetes-pods) however, I think there is something missing in the design and as such it should be posted here.

I have tried this in Kubernetes 1.5
Given a SecurityContext for a pod, a volume to mount and a nested path as the mount point one can clearly see that the intermediate folders in the mount path are left to their own devices as far as permissions are concerned. Allow me to demonstrate:

pod security context:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000

volumeMounts:
    - name: sec-ctx-vol
      mountPath: /data/storage/mount/demo

volumes:
  - name: sec-ctx-vol
    emptyDir: {}

Resulting folder permissions:

$kubectl exec perm-demo -n play -- ls -lR /data
/data:
total 4
drwxr-xr-x    3 root     root          4096 Aug 20 17:45 storage

/data/storage:
total 4
drwxr-xr-x    3 root     root          4096 Aug 20 17:45 mount

/data/storage/mount:
total 4
drwxrwsrwx    2 root     2000          4096 Aug 20 17:45 demo

/data/storage/mount/demo:
total 0

As you can see other than the final folder (demo) all other folders are still owned by root:root in a way that my pod/container user will have no write permissions to them.

So if my pod/container user were to say create a folder like /data/storage/folder2 this would fail since neither runAsGroup nor root groups have any kind of write permissions to that path!

The text was updated successfully, but these errors were encountered:

jeffsaremi · 2019-08-20T19:00:20Z

/sig architecture

zouyee · 2019-08-21T05:52:15Z

/sig storage

liggitt · 2019-08-21T11:11:16Z

/remove-sig architecture

jeffsaremi · 2019-08-22T02:04:47Z

There seems to be at least one way to avoid this problem.

I went back to the same spec and this time added a top-level mount point as well as a deeper one.
Now all the intermediate folders got their permissions from the desirable top one:

pod.yaml (checkout data2 and where it's mounted):

apiVersion: v1
kind: Pod
metadata:
  name: perm-pod
  namespace: play
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 2000
    fsGroup: 2000
  containers:
  - name: permtest
    image: ubuntu:16.04
    command: [ 'sh', '-c', 'sleep 1h' ]
    volumeMounts:
    - name: data2
      mountPath: /data
    - name: data
      mountPath: /data/storage/sample/one
  volumes:
  - name: data
    emptyDir: {}
  - name: data2
    emptyDir: {}

After applying and running a query I get:

$ kubectl exec perm-pod -n play -- sh -c " ls -ld /data; ls -lR /data"
drwxrwsrwx 3 root 2000 4096 Aug 22 01:59 /data
/data:
total 4
drwxr-sr-x 3 root 2000 4096 Aug 22 01:59 storage

/data/storage:
total 4
drwxr-sr-x 3 root 2000 4096 Aug 22 01:59 sample

/data/storage/sample:
total 4
drwxrwsrwx 2 root 2000 4096 Aug 22 01:59 one

/data/storage/sample/one:
total 0

As you can see the "storage" and "sample" also got their permissions inherited from the top-most level (/data)

fejta-bot · 2019-11-20T02:52:45Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot · 2019-12-20T03:35:25Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

fejta-bot · 2020-01-19T04:21:28Z

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

k8s-ci-robot · 2020-01-19T04:21:37Z

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

In a statefulset aggregator config, we were observing errors like this: 2024-01-23T16:57:12.710877825Z ERR Ingestor: insert error: failed to write parquet for file federated/qa-gcp1/etl/bingen/allocations/1d/1705363200-1705449600: error writing container parquet file: error making directory /var/configs/waterfowl/parquet/kubernetes/qa-gcp1/container/1d: mkdir /var/configs/waterfowl/parquet: permission denied Some debugging commands to add context: → k exec -it -n kubecost-cloudcost kubecost-cloudcost-aggregator-0 -- mkdir /var/configs/waterfowl/parquet mkdir: cannot create directory '/var/configs/waterfowl/parquet': Permission denied command terminated with exit code 1 → k exec -it -n kubecost-cloudcost kubecost-cloudcost-aggregator-0 -- ls -alh /var/configs total 68K drwxrwsr-x 5 root 1001 4.0K Jan 18 10:50 . drwxr-xr-x 1 root root 4.0K Jan 23 16:52 .. -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 advanced-reports.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 asset-reports.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 budgets.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 cloud-cost-reports.json -rw-r--r-- 1 1001 1001 112 Jan 23 00:36 collections.json drwxrwsrwt 3 root 1001 100 Jan 23 16:52 etl -rw-r--r-- 1 1001 1001 378 Jan 23 16:52 group-reports.json drwxrws--- 2 root 1001 16K Dec 21 21:23 lost+found -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 recurring-budget-rules.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 reports.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 teams.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 users.json drwxr-sr-x 3 root 1001 4.0K Jan 22 19:05 waterfowl (note that the /var/configs/waterfowl directory is owned by user root and the container process' group (100) does not have write permission, this is what I suspect the source of the error is) This issue seems to back up the theory: kubernetes/kubernetes#81676 Signed-off-by: Michael Dresser <michaelmdresser@gmail.com>

In a statefulset aggregator config, we were observing errors like this: 2024-01-23T16:57:12.710877825Z ERR Ingestor: insert error: failed to write parquet for file federated/qa-gcp1/etl/bingen/allocations/1d/1705363200-1705449600: error writing container parquet file: error making directory /var/configs/waterfowl/parquet/kubernetes/qa-gcp1/container/1d: mkdir /var/configs/waterfowl/parquet: permission denied Some debugging commands to add context: → k exec -it -n kubecost-cloudcost kubecost-cloudcost-aggregator-0 -- mkdir /var/configs/waterfowl/parquet mkdir: cannot create directory '/var/configs/waterfowl/parquet': Permission denied command terminated with exit code 1 → k exec -it -n kubecost-cloudcost kubecost-cloudcost-aggregator-0 -- ls -alh /var/configs total 68K drwxrwsr-x 5 root 1001 4.0K Jan 18 10:50 . drwxr-xr-x 1 root root 4.0K Jan 23 16:52 .. -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 advanced-reports.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 asset-reports.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 budgets.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 cloud-cost-reports.json -rw-r--r-- 1 1001 1001 112 Jan 23 00:36 collections.json drwxrwsrwt 3 root 1001 100 Jan 23 16:52 etl -rw-r--r-- 1 1001 1001 378 Jan 23 16:52 group-reports.json drwxrws--- 2 root 1001 16K Dec 21 21:23 lost+found -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 recurring-budget-rules.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 reports.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 teams.json -rw-r--r-- 1 1001 1001 2 Jan 23 16:52 users.json drwxr-sr-x 3 root 1001 4.0K Jan 22 19:05 waterfowl (note that the /var/configs/waterfowl directory is owned by user root and the container process' group (100) does not have write permission, this is what I suspect the source of the error is) This issue seems to back up the theory: kubernetes/kubernetes#81676 Signed-off-by: Michael Dresser <michaelmdresser@gmail.com> Co-authored-by: Michael Dresser <michaelmdresser@gmail.com>

jeffsaremi added the kind/bug Categorizes issue or PR as related to a bug. label Aug 20, 2019

k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Aug 20, 2019

k8s-ci-robot added sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 20, 2019

k8s-ci-robot added the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Aug 21, 2019

k8s-ci-robot removed the sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. label Aug 21, 2019

h0tbird mentioned this issue Aug 28, 2019

securityContext fsGroup has no effect rancher/local-path-provisioner#41

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 20, 2019

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 20, 2019

k8s-ci-robot closed this as completed Jan 19, 2020

michaelmdresser mentioned this issue Jan 23, 2024

Create emptyDir to fix permissions problem on /var/configs/waterfowl in Aggregator StatefulSet configuration kubecost/cost-analyzer-helm-chart#2953

Merged

gcp-cherry-pick-bot bot mentioned this issue Jan 23, 2024

Create emptyDir to fix permissions problem on /var/configs/waterfowl in Aggregator StatefulSet configuration (cherry-pick #2953) kubecost/cost-analyzer-helm-chart#2956

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

There seems to be no way of setting permissions on the mount path #81676

There seems to be no way of setting permissions on the mount path #81676

jeffsaremi commented Aug 20, 2019

jeffsaremi commented Aug 20, 2019

zouyee commented Aug 21, 2019

liggitt commented Aug 21, 2019

jeffsaremi commented Aug 22, 2019

fejta-bot commented Nov 20, 2019

fejta-bot commented Dec 20, 2019

fejta-bot commented Jan 19, 2020

k8s-ci-robot commented Jan 19, 2020

There seems to be no way of setting permissions on the mount path #81676

There seems to be no way of setting permissions on the mount path #81676

Comments

jeffsaremi commented Aug 20, 2019

jeffsaremi commented Aug 20, 2019

zouyee commented Aug 21, 2019

liggitt commented Aug 21, 2019

jeffsaremi commented Aug 22, 2019

fejta-bot commented Nov 20, 2019

fejta-bot commented Dec 20, 2019

fejta-bot commented Jan 19, 2020

k8s-ci-robot commented Jan 19, 2020