PV not created

[ryanezc]

Hi,
When I deploy rasa using kubernetes and helm, it does not create PV so there are pods that get stuck in the pending state.

NAME READY STATUS RESTARTS AGE
rasa-x-app-5b95d5f5bf-f4xhh 0/1 Running 0 14s
rasa-x-db-migration-service-0 0/1 Running 0 14s
rasa-x-duckling-66676d4f65-z7xjm 0/1 Running 0 14s
rasa-x-event-service-67df4b87c6-hqq7g 0/1 Init:0/1 0 14s
rasa-x-nginx-7f7f7fbcf9fc-985gv 0/1 Running 0 14s
rasa-x-postgresql-0 0/1 Pending 0 14s
rasa-x-rabbit-0 0/1 Pending 0 14s
rasa-x-rasa-worker-5bf97689b-jldzl 0/1 Init:0/1 0 14s
rasa-x-rasa-x-74d6c866ff-5zsq6 0/1 Pending 0 14s
rasa-x-redis-master-0 0/1 Pending 0 14s

why is the pv not created? is there a documented manual procedure?

Thanks

[sara-tagger]

Thanks for raising this issue, @euklein will get back to you about it soon✨

Please also check out the docs and the forum in case your issue was raised there too 🤗

[RASADSA]

Hello @ryanezc
assuming you have a default Storage class set in your k8s cluster all should work.
check the events in the rasa namespace for errors from the k8s deployment
kubectl -n <rasa namespace> get events

[ryanezc]

Hello @RASADSA

Yes, I have a default sc:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
local-storage (default) kubernetes.io/no-provisioner Delete Immediate false 20h

and kubectl --namespace gwydion-andy-rasa get events :
LAST SEEN TYPE REASON OBJECT MESSAGE
3m1s Warning ProvisioningFailed persistentvolumeclaim/data-rasa-x-postgresql-0 no volume plugin matched name: kubernetes.io/no-provisioner

3m1s Warning ProvisioningFailed persistentvolumeclaim/data-rasa-x-rabbit-0 no volume plugin matched name: kubernetes.io/no-provisioner

23m Warning Unhealthy pod/rasa-x-db-migration-service-0 Liveness probe failed: HTTP probe failed with statuscode: 500

3m Warning BackOff pod/rasa-x-db-migration-service-0 Back-off restarting failed container

2m13s Warning FailedScheduling pod/rasa-x-postgresql-0 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.

2m13s Warning FailedScheduling pod/rasa-x-rabbit-0 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.

3m Warning Unhealthy pod/rasa-x-rasa-x-74d6c866ff-nwb99 Liveness probe failed: Get "http://192.168.7.135:5002/api/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

28m Warning Unhealthy pod/rasa-x-rasa-x-74d6c866ff-nwb99 Readiness probe failed: Get "http://192.168.7.135:5002/api/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

7m59s Warning BackOff pod/rasa-x-rasa-x-74d6c866ff-nwb99 Back-off restarting failed container

2m13s Warning FailedScheduling pod/rasa-x-redis-master-0 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.

3m1s Warning ProvisioningFailed persistentvolumeclaim/redis-data-rasa-x-redis-master-0 no volume plugin matched name: kubernetes.io/no-provisioner

I'm going to try to do it manually but i have no clear how to proceed...I just arrived to K8s a week ago.

Thank you for your time in advance.
BR
Rafael

[RASADSA]

Hello @ryanezc

your K8S CSI has no provisioner configured it seems.

3m1s Warning ProvisioningFailed persistentvolumeclaim/data-rasa-x-postgresql-0 no volume plugin matched name: kubernetes.io/no-provisioner

i can only point you in some directions since i have not enough information on what K8S you are using and how its operated.

if you are running on a single node cluster k8s you can use the local path provisioner

https://github.com/rancher/local-path-provisioner

or if you have a multinode kubernetes cluster either use the Cloud Provider K8S CSI solutions by that provider or switch to as example

rook - https://rook.io/docs/rook/v1.8/quickstart.html
openebs - https://openebs.io/docs

another thing to point you in learning K8S in an better experience is

KIND which uses docker to create for you a local cluster to learn the K8S Eco System

https://kind.sigs.k8s.io/docs/user/quick-start/

RASA create REI wto bootstrap with KIND a local K8S Enviroment and ueses RASA rasactl to create easy rasa x deployments to learn demo train on.

https://github.com/RasaHQ/REI
https://github.com/RasaHQ/rasactl

hope this helps to get you started

[ryanezc]

Hi @RASADSA

Thank you VERY MUCH for all this information :-)

Looks like it is more about my lack of knowledge in this moment about K8S PV & PVCs...In fact yesterday already tried with openebs, and all PVCs got bounded. However I got another error:
Output: mount: /var/lib/kubelet/pods/c1011fa6-b794-483e-943f-d2685a5577a3/volumes/kubernetes.io~nfs/pvc-3795af76-7567-4a77-a02c-239df6e5bb69: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount. helper program.

I will continue to research and study on my own.

So again thank you very much for your time and help!!! You might close this issue. When I find the solution I will add it here as a comment in case it can be of help to someone as clueless as me ;-)
BR
Rafael