[FEATURE REQUEST] In Epiphany 2.0.0 StatefulSet 'rabbitmq-cluster' is not set up for persistence

[toszo]

Describe the bug
StatefulSet's "volumeMounts" should use "volumeClaimTemplates", like in this Kubernetes doc https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ or in this example https://gist.github.com/pmint93/cb87c5f46502ce8047a084238cad03e4, to mount "/var/lib/rabbitmq/mnesia" to a PersistentVolume.
At the moment, all that's configured is a volume of type configMap, that injects RMQ configuration into the running container, and that's not enough to avoid losing RMQ configuration and data when the RMQ pod gets moved or the node hosting the RMQ pod gets shut down.

To Reproduce

1. Create Epiphany with RabbitMQ cluster on top of K8s
2. Add users, permissions, etc. to RMQ
3. Shut down Kubernetes nodes that host the RMQ pods, then restart them.
4. Result: any RMQ configuration done on top of the default one, and all the RMQ data are lost.

Expected behavior
Data on RabbitMQ should be persistent and work between restarts.

Config files
If applicable, add config files to help explain your problem.

OS (please complete the following information):

• OS: [e.g. RHEL7]

Additional context
Teams request, EP-118

Suggestion from creator

So in the StatefulSet configuration, in addition to this:
volumeMounts:

mountPath: /etc/rabbitmq
name: config-volume
volumes:

configMap:
defaultMode: 420
items:
- key: rabbitmq.conf
path: rabbitmq.conf
- key: enabled_plugins
path: enabled_plugins
name: rabbitmq-config
name: config-volume

add something like this (dependent on storage provider):
volumeMounts:

name: rabbitmq-data
mountPath: /var/lib/rabbitmq/mnesia
volumeClaimTemplates:

metadata:
name: rabbitmq-data
annotations:
volume.beta.kubernetes.io/storage-class: gp2
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 4Gi

[toszo]

Consider using helm chart for this issue.

[mkyc]

This is duplicate of #1486. @toszo I'd like to close this task and add comment about more context to that another one. Is that ok?

[toszo]

Sounds good.

[seriva]

Closing as all information was moved to #1486