disable rbd plugin on one node

[manju-rn]

I have 3 node microk8s cluster - all are LXD based, 2 are VM and 1 is a container - All are Ubuntu 23.10
The 2 VMs work fine and csi-rbdplugin-xxxx works fine
The 1 LXC container has CrashLoopback for rbd pods and the error is:
csi-rbdplugin-vbql8 0/2 CrashLoopBackOff
Error: modprobe args: [rbd]): "modprobe: FATAL: Module rbd not found in directory /lib/modules/6.2.0-39-generic\n"

The LXC container does not have rbd module and i am okay with that since the 2 VM nodes work fine.
How can i disable the running of csi-rbdplugin in the LXC container. Which option I should choose in operator.yaml to allow only running in VM nodes?

[Madhu-1]

@manju-rn you can use taints/tolerations to customize the rbdplugin

rook/deploy/examples/operator.yaml

Lines 204 to 214 in c409a6a

	# CSI_PLUGIN_NODE_AFFINITY: "role=storage-node; storage=rook, ceph"
	# (Optional) CephCSI plugin tolerations list(applied to both CephFS and RBD plugin).
	# Put here list of taints you want to tolerate in YAML format.
	# CSI plugins need to be started on all the nodes where the clients need to mount the storage.
	# CSI_PLUGIN_TOLERATIONS: |
	# - effect: NoSchedule
	# key: node-role.kubernetes.io/control-plane
	# operator: Exists
	# - effect: NoExecute
	# key: node-role.kubernetes.io/etcd
	# operator: Exists

[manju-rn]

@manju-rn you can use taints/tolerations to customize the rbdplugin

rook/deploy/examples/operator.yaml

Lines 204 to 214 in c409a6a

	# CSI_PLUGIN_NODE_AFFINITY: "role=storage-node; storage=rook, ceph"
	# (Optional) CephCSI plugin tolerations list(applied to both CephFS and RBD plugin).
	# Put here list of taints you want to tolerate in YAML format.
	# CSI plugins need to be started on all the nodes where the clients need to mount the storage.
	# CSI_PLUGIN_TOLERATIONS: |
	# - effect: NoSchedule
	# key: node-role.kubernetes.io/control-plane
	# operator: Exists
	# - effect: NoExecute
	# key: node-role.kubernetes.io/etcd
	# operator: Exists

Thanks. help me understand, this will stop even the FS and Objectorage plugin also? I want only the RBD plugin to stop, since Objectstorage works on LXC container

[Madhu-1]

in that case you can use

rook/deploy/examples/operator.yaml

Lines 228 to 231 in c409a6a

	# CSI plugins need to be started on all the nodes where the clients need to mount the storage.
	# CSI_RBD_PLUGIN_TOLERATIONS: |
	# - key: node.rook.io/rbd
	# operator: Exists

its specific to RBD plugin.

[manju-rn]

Thanks. So just for my understanding
This is for both CephFS and RBD plugin

Lines 204 to 214 in c409a6a

This is for only RBD Plugin

Lines 228 to 231 in c409a6a

And this is for FS

rook/deploy/examples/operator.yaml

Line 245 in c409a6a

# CSI_CEPHFS_PLUGIN_NODE_AFFINITY: |

and so on.

[Madhu-1]

Yes correct, Closing this one, Feel free to reopen if it didn't work