Remove subvolumegroup creation from cephcsi

[Madhu-1]

I don't think it's a good idea for us to create subvolumegroups at cephcsi. It should be the responsibility of the ceph admin to handle this task. Instead, cephcsi should only create subvolumes that correspond to the PVC/PV.
Additionally, we should avoid including pin details in the storageclass or clusterID configmap since a single subvolumegroup might be used by multiple storageclasses/clusterIDs. Pinning details are subject to change and should be considered a dynamic configuration.
Lastly, it's best if the ceph admin manages the creation and management of the filesystem and subvolumegroup.

Originally posted by @Madhu-1 in #2637 (comment)

[iPraveenParihar]

/assign

[parth-gr]

@Madhu-1 so with this change rook should always create the csi subvolumegroup while creating filesystem?

[Madhu-1]

@Madhu-1 so with this change rook should always create the csi subvolumegroup while creating filesystem?

It should be created by Rook but I assume if you add it to filesystem.yaml file(s) for the user will get it created automatically.

[travisn]

In the common cases this sounds good:

1. Upgraded clusters would already have the csi subvolumegroup created, so no issues during upgrade
2. New clusters would create the subvolumegroup with the filesystem.yaml

What is the behavior if the csi subvolumegroup is not created for some reason? Will provisioning the cephfs volume just fail? I wonder if Rook should create it by default when the filesystem is created, in case the CR is missed.

[Madhu-1]

In the common cases this sounds good:

1. Upgraded clusters would already have the csi subvolumegroup created, so no issues during upgrade
2. New clusters would create the subvolumegroup with the filesystem.yaml

What is the behavior if the csi subvolumegroup is not created for some reason? Will provisioning the cephfs volume just fail? I wonder if Rook should create it by default when the filesystem is created, in case the CR is missed.

@travisn if the subvolumegroup is missing the PVC creation will fail and PVC will stay in Pending state. sounds good to create the subvolumegroup by default when a filesystem is created. and if any customization is required a user can create the CR and override it as well. WDYT?

[travisn]

In the common cases this sounds good:

1. Upgraded clusters would already have the csi subvolumegroup created, so no issues during upgrade
2. New clusters would create the subvolumegroup with the filesystem.yaml

What is the behavior if the csi subvolumegroup is not created for some reason? Will provisioning the cephfs volume just fail? I wonder if Rook should create it by default when the filesystem is created, in case the CR is missed.

@travisn if the subvolumegroup is missing the PVC creation will fail and PVC will stay in Pending state. sounds good to create the subvolumegroup by default when a filesystem is created. and if any customization is required a user can create the CR and override it as well. WDYT?

Agreed sounds good to me to create it by default if the CR is missing.