csi: option to customize csi driver name prefix

For now we are using the operator namespace name
as the prefix for the csi driver, This PR provides
an option for the users if someone wants to have
their own prefix for the csi driver, if someone tries
to change the prefix for existing csi driver rook
operator will fail to reconcile the csi driver.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>

Documentation/Helm-Charts/ceph-cluster-chart.md

@@ -69,6 +69,7 @@ The following table lists the configurable parameters of the rook-operator chart
 | `cephObjectStores` | A list of CephObjectStore configurations to deploy | See [below](#ceph-object-stores) |
 | `clusterName` | The metadata.name of the CephCluster CR | The same as the namespace |
 | `configOverride` | Cluster ceph.conf override | `nil` |
+| `csiDriverNamePrefix` | CSI driver name prefix for cephfs, rbd and nfs. | `namespace name where rook-ceph operator is deployed` |
 | `ingress.dashboard` | Enable an ingress for the ceph-dashboard | `{}` |
 | `kubeVersion` | Optional override of the target kubernetes version | `nil` |
 | `monitoring.createPrometheusRules` | Whether to create the Prometheus rules for Ceph alerts | `false` |

Documentation/Helm-Charts/operator-chart.md

@@ -70,6 +70,7 @@ The following table lists the configurable parameters of the rook-operator chart
 | `csi.csiCephFSPluginVolume` | The volume of the CephCSI CephFS plugin DaemonSet | `nil` |
 | `csi.csiCephFSPluginVolumeMount` | The volume mounts of the CephCSI CephFS plugin DaemonSet | `nil` |
 | `csi.csiCephFSProvisionerResource` | CEPH CSI CephFS provisioner resource requirement list | see values.yaml |
+| `csi.csiDriverNamePrefix` | CSI driver name prefix for cephfs, rbd and nfs. | `namespace name where rook-ceph operator is deployed` |
 | `csi.csiLeaderElectionLeaseDuration` | Duration in seconds that non-leader candidates will wait to force acquire leadership. | `137s` |
 | `csi.csiLeaderElectionRenewDeadline` | Deadline in seconds that the acting leader will retry refreshing leadership before giving up. | `107s` |
 | `csi.csiLeaderElectionRetryPeriod` | Retry period in seconds the LeaderElector clients should wait between tries of actions. | `26s` |

Documentation/Storage-Configuration/Advanced/ceph-configuration.md

@@ -45,14 +45,17 @@ sed -i.bak \
     -e "s/\(.*\):.*# namespace:cluster/\1: $ROOK_CLUSTER_NAMESPACE # namespace:cluster/g" \
     -e "s/\(.*serviceaccount\):.*:\(.*\) # serviceaccount:namespace:operator/\1:$ROOK_OPERATOR_NAMESPACE:\2 # serviceaccount:namespace:operator/g" \
     -e "s/\(.*serviceaccount\):.*:\(.*\) # serviceaccount:namespace:cluster/\1:$ROOK_CLUSTER_NAMESPACE:\2 # serviceaccount:namespace:cluster/g" \
-    -e "s/\(.*\): [-_A-Za-z0-9]*\.\(.*\) # driver:namespace:operator/\1: $ROOK_OPERATOR_NAMESPACE.\2 # driver:namespace:operator/g" \
     -e "s/\(.*\): [-_A-Za-z0-9]*\.\(.*\) # driver:namespace:cluster/\1: $ROOK_CLUSTER_NAMESPACE.\2 # driver:namespace:cluster/g" \
   common.yaml operator.yaml cluster.yaml # add other files or change these as desired for your config
 
 # You need to use `apply` for all Ceph clusters after the first if you have only one Operator
 kubectl apply -f common.yaml -f operator.yaml -f cluster.yaml # add other files as desired for yourconfig
 ```
 
+Refer to the CSI driver
+[documention](../Ceph-CSI/ceph-csi-drivers.md#Configure-CSI-Drivers-in-non-default-namespace)
+to update examples to use a non-default namespace.
+
 ## Deploying a second cluster
 
 If you wish to create a new CephCluster in a separate namespace, you can easily do so

Documentation/Storage-Configuration/Ceph-CSI/ceph-csi-drivers.md

@@ -35,6 +35,87 @@ example, if the Rook operator is running in the namespace `my-namespace` the
 provisioner value should be `my-namespace.rbd.csi.ceph.com`. The same provisioner
 name must be set in both the storageclass and snapshotclass.
 
+search for `# csi-provisioner-name` in the [csi
+example](../deploy/examples/csi/) folder and update the name of the
+`provisioner`/`driver` as required.
+
+The following `sed` command can be used to update the provisioner name in the
+csi example manifests:
+
+```console
+cd deploy/example/csi
+
+export ROOK_OPERATOR_NAMESPACE=rook-ceph
+sed -i.bak \
+    -e "s/\(.*\): [-_A-Za-z0-9]*\.\(.*\) # csi-provisioner-name/\1: $ROOK_OPERATOR_NAMESPACE.\2 # csi-provisioner-name/g" \
+    rbd/*storageclass*.yaml rbd/*snapshotclass*.yaml cephfs/*storageclass*.yaml cephfs/*snapshotclass*.yaml \
+    nfs/*storageclass*.yaml nfs/*snapshotclass*.yaml
+```
+
+## Configure custom Driver name prefix for CSI Drivers
+
+To use a custom prefix for the CSI drivers instead of the namespace prefix, set
+the `CSI_DRIVER_NAME_PREFIX` environment variable in the operator configmap.
+For instance, to use the prefix `my-prefix` for the CSI drivers, set
+the following in the operator configmap:
+
+```console
+kubectl patch cm rook-ceph-operator-config -n rook-ceph -p $'data:\n "CSI_DRIVER_NAME_PREFIX": "my-prefix"'
+```
+
+Once the configmap is updated, the CSI drivers will be deployed with the
+`my-prefix` prefix. The same prefix must be set in both the storageclass and
+snapshotclass. For example, to use the prefix `my-prefix` for the
+CSI drivers, you can set the following in the storageclass:
+
+```yaml
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: rook-ceph-block-sc
+provisioner: my-prefix.rbd.csi.ceph.com
+...
+```
+
+The same prefix must be set in the volumesnapshotclass as well:
+
+```yaml
+apiVersion: snapshot.storage.k8s.io/v1
+kind: VolumeSnapshotClass
+metadata:
+  name: rook-ceph-block-vsc
+driver: my-prefix.rbd.csi.ceph.com
+...
+```
+
+When the prefix is set, the driver names will be:
+
+* RBD: `my-prefix.rbd.csi.ceph.com`
+* CephFS: `my-prefix.cephfs.csi.ceph.com`
+* NFS: `my-prefix.nfs.csi.ceph.com`
+
+!!! note
+    Please be careful when setting the `CSI_DRIVER_NAME_PREFIX`
+    environment variable. It should be done only in fresh deployments because
+    changing the prefix in an existing cluster will result in unexpected behavior.
+
+search for `# csi-provisioner-name` in the [csi
+example](../deploy/examples/csi/) folder and update the name of the
+`provisioner`/`driver` as required.
+
+The following `sed` command can be used to update the provisioner name in the
+csi example manifests:
+
+```console
+cd deploy/example/csi
+
+export CSI_DRIVER_NAME_PREFIX=my-prefix
+sed -i.bak \
+    -e "s/\(.*\): [-_A-Za-z0-9]*\.\(.*\) # csi-provisioner-name/\1: $CSI_DRIVER_NAME_PREFIX.\2 # csi-provisioner-name/g" \
+    rbd/*storageclass*.yaml rbd/*snapshotclass*.yaml cephfs/*storageclass*.yaml cephfs/*snapshotclass*.yaml \
+    nfs/*storageclass*.yaml nfs/*snapshotclass*.yaml
+```
+
 ## Liveness Sidecar
 
 All CSI pods are deployed with a sidecar container that provides a Prometheus
@@ -116,7 +197,6 @@ Refer to the [ephemeral-doc](https://kubernetes.io/docs/concepts/storage/ephemer
 See example manifests for an [RBD ephemeral volume](https://github.com/rook/rook/tree/master/deploy/examples/csi/rbd/pod-ephemeral.yaml)
 and a [CephFS ephemeral volume](https://github.com/rook/rook/tree/master/deploy/examples/csi/cephfs/pod-ephemeral.yaml).
 
-
 ## CSI-Addons Controller
 
 The CSI-Addons Controller handles requests from users. Users create a CR

deploy/charts/rook-ceph-cluster/templates/cephblockpool.yaml

@@ -16,7 +16,11 @@ metadata:
   name: {{ $blockpool.storageClass.name }}
   annotations:
     storageclass.kubernetes.io/is-default-class: "{{ if default false $blockpool.storageClass.isDefault }}true{{ else }}false{{ end }}"
+{{- if $root.Values.csiDriverNamePrefix }}
+provisioner: {{ $root.Values.csiDriverNamePrefix }}.rbd.csi.ceph.com
+{{- else }}
 provisioner: {{ $root.Values.operatorNamespace }}.rbd.csi.ceph.com
+{{- end }}
 parameters:
   pool: {{ $blockpool.name }}
   clusterID: {{ $root.Release.Namespace }}

deploy/charts/rook-ceph-cluster/templates/cephecblockpool.yaml

@@ -16,7 +16,11 @@ apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: {{ $cephEcStorage.name }}
-provisioner: {{ $cephEcStorage.provisioner }}
+{{- if $root.Values.csiDriverNamePrefix }}
+provisioner: {{ $root.Values.csiDriverNamePrefix }}.rbd.csi.ceph.com
+{{- else }}
+provisioner: {{ $root.Values.operatorNamespace }}.rbd.csi.ceph.com
+{{- end }}
 parameters:
   clusterID: {{ $cephEcStorage.parameters.clusterID }}
   dataPool: {{ $cephEcStorage.parameters.dataPool }}

deploy/charts/rook-ceph-cluster/templates/cephfilesystem.yaml

@@ -35,7 +35,11 @@ metadata:
   name: {{ $filesystem.storageClass.name }}
   annotations:
     storageclass.kubernetes.io/is-default-class: "{{ if default false $filesystem.storageClass.isDefault }}true{{ else }}false{{ end }}"
+{{- if $root.Values.csiDriverNamePrefix }}
+provisioner: {{ $root.Values.csiDriverNamePrefix }}.cephfs.csi.ceph.com
+{{- else }}
 provisioner: {{ $root.Values.operatorNamespace }}.cephfs.csi.ceph.com
+{{- end }}
 parameters:
   fsName: {{ $filesystem.name }}
   pool: {{ $filesystem.name }}-{{ default "data0" $filesystem.storageClass.pool }}

deploy/charts/rook-ceph-cluster/templates/volumesnapshotclass.yaml

@@ -1,3 +1,4 @@
+{{- $root := . -}}
 {{- $filesystemvsc := .Values.cephFileSystemVolumeSnapshotClass -}}
 {{- $blockpoolvsc := .Values.cephBlockPoolsVolumeSnapshotClass -}}
 
@@ -16,7 +17,11 @@ metadata:
 {{- if $filesystemvsc.annotations }}
 {{ toYaml $filesystemvsc.annotations | indent 4 }}
 {{- end }}
-driver: {{ .Values.operatorNamespace }}.cephfs.csi.ceph.com
+{{- if $root.Values.csiDriverNamePrefix }}
+driver: {{ $root.Values.csiDriverNamePrefix }}.cephfs.csi.ceph.com
+{{- else }}
+driver: {{ $root.Values.operatorNamespace }}.cephfs.csi.ceph.com
+{{- end }}
 parameters:
   clusterID: {{ .Release.Namespace }}
   csi.storage.k8s.io/snapshotter-secret-name: rook-csi-cephfs-provisioner
@@ -42,7 +47,11 @@ metadata:
 {{- if $blockpoolvsc.annotations }}
 {{ toYaml $blockpoolvsc.annotations | indent 4 }}
 {{- end }}
-driver: {{ .Values.operatorNamespace }}.rbd.csi.ceph.com
+{{- if $root.Values.csiDriverNamePrefix }}
+driver: {{ $root.Values.csiDriverNamePrefix }}.rbd.csi.ceph.com
+{{- else }}
+driver: {{ $root.Values.operatorNamespace }}.rbd.csi.ceph.com
+{{- end }}
 parameters:
   clusterID: {{ .Release.Namespace }}
   csi.storage.k8s.io/snapshotter-secret-name: rook-csi-rbd-provisioner

deploy/charts/rook-ceph-cluster/values.yaml

@@ -647,8 +647,6 @@ cephObjectStores:
 # if cephECBlockPools are uncommented you must remove the comments of cephEcStorageClass as well
 #cephECStorageClass:
 #  name: rook-ceph-block
-#  # Change "rook-ceph" provisioner prefix to match the operator namespace if needed
-#  provisioner: rook-ceph.rbd.csi.ceph.com # driver:namespace:operator
 #  parameters:
 #    # clusterID is the namespace where the rook cluster is running
 #    # If you change this namespace, also change the namespace below where the secret namespaces are defined
@@ -687,3 +685,7 @@ cephObjectStores:
 #    imageFeatures: layering
 #  allowVolumeExpansion: true
 #  reclaimPolicy: Delete
+
+# -- CSI driver name prefix for cephfs, rbd and nfs.
+# @default -- `namespace name where rook-ceph operator is deployed`
+csiDriverNamePrefix:

deploy/charts/rook-ceph/templates/configmap.yaml

@@ -25,6 +25,9 @@ data:
   CSI_ENABLE_OMAP_GENERATOR: {{ .Values.csi.enableOMAPGenerator | quote }}
   CSI_ENABLE_HOST_NETWORK: {{ .Values.csi.enableCSIHostNetwork | quote }}
   CSI_ENABLE_METADATA: {{ .Values.csi.enableMetadata | quote }}
+{{- if .Values.csi.csiDriverNamePrefix }}
+  CSI_DRIVER_NAME_PREFIX: {{ .Values.csi.csiDriverNamePrefix | quote }}
+{{- end }}
 {{- if .Values.csi.pluginPriorityClassName }}
   CSI_PLUGIN_PRIORITY_CLASSNAME: {{ .Values.csi.pluginPriorityClassName | quote }}
 {{- end }}

deploy/charts/rook-ceph/values.yaml

@@ -147,6 +147,10 @@ csi:
   # @default -- `0`
   sidecarLogLevel:
 
+  # -- CSI driver name prefix for cephfs, rbd and nfs.
+  # @default -- `namespace name where rook-ceph operator is deployed`
+  csiDriverNamePrefix:
+
   # -- CSI RBD plugin daemonset update strategy, supported values are OnDelete and RollingUpdate
   # @default -- `RollingUpdate`
   rbdPluginUpdateStrategy:

deploy/examples/csi/cephfs/snapshotclass.yaml

@@ -3,7 +3,10 @@ apiVersion: snapshot.storage.k8s.io/v1
 kind: VolumeSnapshotClass
 metadata:
   name: csi-cephfsplugin-snapclass
-driver: rook-ceph.cephfs.csi.ceph.com # driver:namespace:operator
+# The prefix of the driver name can be configured by updating
+# CSI_DRIVER_NAME_PREFIX in the rook-ceph-operator-config configmap by default
+# it is namepsace name where rook operator is running i.e rook-ceph.
+driver: .cephfs.csi.ceph.com # csi-provisioner-name
 parameters:
   # Specify a string that identifies your cluster. Ceph CSI supports any
   # unique string. When Ceph CSI is deployed by Rook use the Rook namespace,

deploy/examples/csi/cephfs/storageclass-ec.yaml

@@ -2,8 +2,10 @@ apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: rook-cephfs
-# Change "rook-ceph" provisioner prefix to match the operator namespace if needed
-provisioner: rook-ceph.cephfs.csi.ceph.com # driver:namespace:operator
+# The prefix of the driver name can be configured by updating
+# CSI_DRIVER_NAME_PREFIX in the rook-ceph-operator-config configmap by default
+# it is namepsace name where rook operator is running i.e rook-ceph.
+provisioner: .cephfs.csi.ceph.com # csi-provisioner-name
 parameters:
   # clusterID is the namespace where the rook cluster is running
   # If you change this namespace, also change the namespace below where the secret namespaces are defined

deploy/examples/csi/cephfs/storageclass.yaml

@@ -2,8 +2,10 @@ apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: rook-cephfs
-# Change "rook-ceph" provisioner prefix to match the operator namespace if needed
-provisioner: rook-ceph.cephfs.csi.ceph.com # driver:namespace:operator
+# The prefix of the driver name can be configured by updating
+# CSI_DRIVER_NAME_PREFIX in the rook-ceph-operator-config configmap by default
+# it is namepsace name where rook operator is running i.e rook-ceph.
+provisioner: .cephfs.csi.ceph.com # csi-provisioner-name
 parameters:
   # clusterID is the namespace where the rook cluster is running
   # If you change this namespace, also change the namespace below where the secret namespaces are defined

deploy/examples/csi/nfs/snapshotclass.yaml

@@ -3,7 +3,10 @@ apiVersion: snapshot.storage.k8s.io/v1
 kind: VolumeSnapshotClass
 metadata:
   name: csi-nfsplugin-snapclass
-driver: rook-ceph.nfs.csi.ceph.com # driver:namespace:operator
+# The prefix of the driver name can be configured by updating
+# CSI_DRIVER_NAME_PREFIX in the rook-ceph-operator-config configmap by default
+# it is namepsace name where rook operator is running i.e rook-ceph.
+driver: rook-ceph.nfs.csi.ceph.com # csi-provisioner-name
 parameters:
   # Specify a string that identifies your cluster. Ceph CSI supports any
   # unique string. When Ceph CSI is deployed by Rook use the Rook namespace,

deploy/examples/csi/rbd/snapshotclass.yaml

@@ -3,7 +3,10 @@ apiVersion: snapshot.storage.k8s.io/v1
 kind: VolumeSnapshotClass
 metadata:
   name: csi-rbdplugin-snapclass
-driver: rook-ceph.rbd.csi.ceph.com # driver:namespace:operator
+# The prefix of the driver name can be configured by updating
+# CSI_DRIVER_NAME_PREFIX in the rook-ceph-operator-config configmap by default
+# it is namepsace name where rook operator is running i.e rook-ceph.
+driver: rook-ceph.rbd.csi.ceph.com # csi-provisioner-name
 parameters:
   # Specify a string that identifies your cluster. Ceph CSI supports any
   # unique string. When Ceph CSI is deployed by Rook use the Rook namespace,

deploy/examples/csi/rbd/storageclass-ec.yaml

@@ -30,8 +30,10 @@ apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: rook-ceph-block
-# Change "rook-ceph" provisioner prefix to match the operator namespace if needed
-provisioner: rook-ceph.rbd.csi.ceph.com # driver:namespace:operator
+# The prefix of the driver name can be configured by updating
+# CSI_DRIVER_NAME_PREFIX in the rook-ceph-operator-config configmap by default
+# it is namepsace name where rook operator is running i.e rook-ceph.
+provisioner: rook-ceph.rbd.csi.ceph.com # csi-provisioner-name
 parameters:
   # clusterID is the namespace where the rook cluster is running
   # If you change this namespace, also change the namespace below where the secret namespaces are defined

deploy/examples/csi/rbd/storageclass-test.yaml

@@ -18,8 +18,10 @@ apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: rook-ceph-block
-# Change "rook-ceph" provisioner prefix to match the operator namespace if needed
-provisioner: rook-ceph.rbd.csi.ceph.com # driver:namespace:operator
+# The prefix of the driver name can be configured by updating
+# CSI_DRIVER_NAME_PREFIX in the rook-ceph-operator-config configmap by default
+# it is namepsace name where rook operator is running i.e rook-ceph.
+provisioner: rook-ceph.rbd.csi.ceph.com # csi-provisioner-name
 parameters:
   # clusterID is the namespace where the rook cluster is running
   # If you change this namespace, also change the namespace below where the secret namespaces are defined

deploy/examples/import-external-cluster.sh

@@ -22,8 +22,9 @@ RBD_STORAGE_CLASS_NAME=ceph-rbd
 CEPHFS_STORAGE_CLASS_NAME=cephfs
 ROOK_EXTERNAL_MONITOR_SECRET=mon-secret
 OPERATOR_NAMESPACE=rook-ceph                                 # default set to rook-ceph
-RBD_PROVISIONER=$OPERATOR_NAMESPACE".rbd.csi.ceph.com"       # driver:namespace:operator
-CEPHFS_PROVISIONER=$OPERATOR_NAMESPACE".cephfs.csi.ceph.com" # driver:namespace:operator
+CSI_DRIVER_NAME_PREFIX=${CSI_DRIVER_NAME_PREFIX:-$OPERATOR_NAMESPACE}
+RBD_PROVISIONER=$CSI_DRIVER_NAME_PREFIX".rbd.csi.ceph.com"       # csi-provisioner-name
+CEPHFS_PROVISIONER=$CSI_DRIVER_NAME_PREFIX=".cephfs.csi.ceph.com" # csi-provisioner-name
 CLUSTER_ID_RBD=$NAMESPACE
 CLUSTER_ID_CEPHFS=$NAMESPACE
 : "${ROOK_EXTERNAL_ADMIN_SECRET:=admin-secret}"

deploy/examples/operator-openshift.yaml

@@ -614,6 +614,10 @@ data:
   # (Optional) Retry period in seconds the LeaderElector clients should wait between tries of actions. Defaults to 26 seconds.
   # CSI_LEADER_ELECTION_RETRY_PERIOD: "26s"
 
+  # csi driver name prefix for cephfs, rbd and nfs. if not specified, default
+  # will be the namespace name where rook-ceph operator is deployed.
+  # CSI_DRIVER_NAME_PREFIX: "rook-ceph"
+
   # Rook Discover toleration. Will tolerate all taints with all keys.
   # (Optional) Rook Discover tolerations list. Put here list of taints you want to tolerate in YAML format.
   # DISCOVER_TOLERATIONS: |

deploy/examples/operator.yaml

@@ -61,6 +61,10 @@ data:
   # Supported values from 0 to 5. 0 for general useful logs (the default), 5 for trace level verbosity.
   # CSI_SIDECAR_LOG_LEVEL: "0"
 
+  # csi driver name prefix for cephfs, rbd and nfs. if not specified, default
+  # will be the namespace name where rook-ceph operator is deployed.
+  # CSI_DRIVER_NAME_PREFIX: "rook-ceph"
+
   # Set replicas for csi provisioner deployment.
   CSI_PROVISIONER_REPLICAS: "2"
 

pkg/operator/ceph/csi/csi.go

@@ -320,5 +320,8 @@ func (r *ReconcileCSI) setParams(ver *version.Info) error {
 	if strings.EqualFold(k8sutil.GetValue(r.opConfig.Parameters, "CSI_NFS_ATTACH_REQUIRED", "true"), "false") {
 		CSIParam.NFSAttachRequired = false
 	}
+
+	CSIParam.DriverNamePrefix = k8sutil.GetValue(r.opConfig.Parameters, "CSI_DRIVER_NAME_PREFIX", r.opConfig.OperatorNamespace)
+
 	return nil
 }