Unable to start osd prepare job due to missing PSP capability

[y1r]

Is this a bug report or feature request?

• Bug Report

Deviation from expected behavior:
Preparing OSD job cannot create pods due to PodSecurityPolicy. Following events are occuring:

Events:
  Type     Reason        Age               From            Message
  ----     ------        ----              ----            -------
  Warning  FailedCreate  6s (x3 over 39s)  job-controller  Error creating: pods "rook-ceph-osd-prepare-set1-data-128zvj--1-" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.initContainers[1].securityC
ontext.capabilities.add: Invalid value: "MKNOD": capability may not be added]

I think adding CAP_MKNOD to 00-rook-priviledged will solve the issue.

Expected behavior:
Preparing OSD pods are created.

How to reproduce it (minimal and precise):

File(s) to submit:

• Cluster CR (custom resource), typically called cluster.yaml, if necessary
• Operator's logs, if necessary
• Crashing pod(s) logs, if necessary

To get logs, use kubectl -n <namespace> logs <pod name>
When pasting logs, always surround them with backticks or use the insert code button from the Github UI.
Read Github documentation if you need help.

Environment:

• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Cloud provider or hardware configuration:
• Rook version (use rook version inside of a Rook Pod): v1.8.0
• Storage backend version (e.g. for ceph do ceph -v):
• Kubernetes version (use kubectl version): Kubernetes 1.22.3
• Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): On-premise with PSP
• Storage backend status (e.g. for Ceph use ceph health in the Rook Ceph toolbox):