Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage addon doesn't work with RBAC #516

Closed
excieve opened this issue Jun 20, 2019 · 2 comments · Fixed by #522
Closed

Storage addon doesn't work with RBAC #516

excieve opened this issue Jun 20, 2019 · 2 comments · Fixed by #522

Comments

@excieve
Copy link

@excieve excieve commented Jun 20, 2019

When RBAC addon is enabled, the hostpath-provisioner pods don't work properly due to missing permissions, hence volume claims can't be created.

Pod logs show this:

E0620 10:14:45.205609       1 reflector.go:201] github.com/juju-solutions/hostpath-provisioner/vendor/github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:295: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0620 10:14:45.206140       1 reflector.go:201] github.com/juju-solutions/hostpath-provisioner/vendor/github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:265: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0620 10:14:45.206954       1 reflector.go:201] github.com/juju-solutions/hostpath-provisioner/vendor/github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:294: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope 

Binding a cluster-admin role to kube-system:default manually "fixes" this, but I believe it should have its own service role and proper permissions when RBAC is enabled.

@ktsakalozos

This comment has been minimized.

Copy link
Member

@ktsakalozos ktsakalozos commented Jun 21, 2019

@excieve we are aware that some of the addons may not work properly with RBAC yet. This is one of these cases. We would like to provide some sane defaults for RBAC permissions and we hope these defaults will develop organically from the users. Would you be able to offer your suggestions here or (even better) in the form of a PR against https://github.com/ubuntu/microk8s/blob/master/microk8s-resources/actions/storage.yaml?

Thank you.

@wichert

This comment has been minimized.

Copy link
Contributor

@wichert wichert commented Jun 26, 2019

@ktsakalozos This should do the trick

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hostpath-provisioner
  labels:
    k8s-app: hostpath-provisioner
  namespace: kube-system
spec:
  replicas: 1
  revisionHistoryLimit: 0
  selector:
    matchLabels:
      k8s-app: hostpath-provisioner
  template:
    metadata:
      labels:
        k8s-app: hostpath-provisioner
    spec:
      serviceAccountName: microk8s-hostpath
      containers:
        - name: hostpath-provisioner
          image: cdkbot/hostpath-provisioner-$ARCH:latest
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: PV_DIR
              value: $SNAP_COMMON/default-storage
#            - name: PV_RECLAIM_POLICY
#              value: Retain
          volumeMounts:
            - name: pv-volume
              mountPath: $SNAP_COMMON/default-storage
      volumes:
        - name: pv-volume
          hostPath:
            path: $SNAP_COMMON/default-storage
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: microk8s-hostpath
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: microk8s.io/hostpath
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: microk8s-hostpath
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: microk8s-hostpath
rules:
- apiGroups: [""]
  resources:
  - persistentvolumeclaims
  verbs:
  - list
  - get
  - watch
  - update
- apiGroups: [""]
  resources:
  - persistentvolumes
  verbs:
  - list
  - get
  - update
  - watch
  - create
- apiGroups: [""]
  resources:
    - events
  verbs:
    - create
    - patch
- apiGroups: ["storage.k8s.io"]
  resources:
    - storageclasses
  verbs:
    - list
    - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: microk8s-hostpath
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: microk8s-hostpath
subjects:
  - kind: ServiceAccount
    name: microk8s-hostpath
    namespace: kube-system
ktsakalozos added a commit that referenced this issue Jun 27, 2019
This fixes #516
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants
You can’t perform that action at this time.