[OpenShift] CrashLoopBackOff when starting DS vault-csi-provider

[devops-42]

Hi,

I tried to install Vault with CSI provider enabled on OpenShift using Helm and the following values.yml:

global:
  openshift: true
server:
  route: 
    enabled: true
ui:
  enabled: true
csi:
  enabled: true
injector:
  enabled: false

After deployment I checked which pods are running in the namespace. Only the vault pod was shown, but no pod for the vault-csi-provider. When looking at the events some SCC issues have occured, so I added the privileged SCC to get the pod started:

oc adm policy add-scc-to-user privileged system:serviceaccount:<vault-namespace>:vault-csi-provider

Now the pod could start, but immediately runs into an error state. The pod's log output is:

[ERROR] Error running provider: err="failed to listen on unix socket at /provider/vault.sock: listen unix /provider/vault.sock: bind: permission denied"

How could this be solved?

Thanks for your help in advance!
Cheers
Matthias

[tomhjp]

I'm not too familiar with OpenShift, but this seems like another permissions issue. The CSI driver and provider communicate over a unix socket within a hostPath volume, and your error suggests that the provider does not have the writeable hostPath volume mount it expects. It seems like the volume is there, but possibly readonly?

[devops-42]

Hi @tomhjp

this was my assumption too. I tried solving it by assigning SCCs to run the CSI pod. Neither:

• privileged
• hostaccess
• hostnetwork
  seem to solve the problem.

[nhat-tong]

Hello,

I ran into this issue today.
@devops-42: did you found a workaround ? I tried to grant some permissions (privileged, hostaccess, hostnetwork) to the sa of vault-csi-provider but it doesn't work at all.
Thanks

[nhat-tong]

@devops-42, @tomhjp: it seems that there is something missing in csi provider manifest for OpenShift.

After patching the daemonset using this line, it fixed the issue:
oc patch daemonset vault-csi-provider --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": {"privileged": true} }]'

Regards,

[tomhjp]

Thanks for the help debugging this. We recently added some documentation to our website to help explain the steps required to install the Vault CSI Provider on OpenShift: https://www.vaultproject.io/docs/platform/k8s/csi/installation#installation-on-openshift

However, just be aware that it requires going against OpenShift's recommendations due to the privileged pods and writeable hostPath volumes required, so we have a notice in that documentation with the same warning. And for the same reason we don't currently plan to add official support in the helm chart. However, there is no such issue with the Agent injector, so hopefully that covers most of the requirements on OpenShift.