CSI-Node-Driver - missing tolerations

[hajdukda]

CSI-Node-Driver missing tolerations. How can we supply them - operator has correct tolerations set - what are consequences of not deploying it on every node ?

Expected Behavior

Operator creates csi-node-driver with tolerations that allow it to run on every node.

Current Behavior

Any taints on the node block csi-node-driver from tigera-operator.

Possible Solution

Some way to configure tigera-operator to apply Exists toleration on DS. Or follow best practices for daemonsets that should run everywhere to use "- operator: Exists" toleration..

Steps to Reproduce (for bugs)

Deploy Calico using tigera-operator 3.24.5

Context

Not sure whether this DS should run on every node together with calico-node.

Your Environment

EKS 1.23

[tmjd]

Yes this is a bug, this #2483 fixes it and it was picked to the branch for v3.24 with #2509.
I'm not sure when an update will be released with the fix.

[hajdukda]

Whats the expected minor version with fix available ? 1.28 tigera operator ?

[tmjd]

I don't see 3.28 anywhere, I'm assuming you are misreading the 1.28. Please correct me if I'm wrong.

The operator v1.28 is used to install Calico v3.24.
That is why #2509 is for r1.28.
Why doesn't the version match? Because this operator is versioned independently from Calico because the open source Calico product is not the only thing it installs.

[tmjd]

It looks like it will be in the operator v1.28.11 image.

[tomsucho]

Hey @tmjd the latest Helm chart that I can see for v3.24.5 line has this:

12:15 $ helm template calico projectcalico/tigera-operator --version=v3.24.5 | grep image:
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/ubuntu/.kube/config
          image: quay.io/tigera/operator:v1.28.5

currently I got v3.24.1 installed with
image: quay.io/tigera/operator:v1.28.1
so can I just override the tag version in values.yaml for the currently used Helm chart (v3.24.1) to v1.28.11, or should I upgrade Helm chart to v3.24.5 first? but then would need to still override the image tag as it only comes with v1.28.5?
Wonder what is the best way to get this fix into my current clusters without affecting things too much, as it seems it is working in general (what would be the consequence of not having CSI-Node-Driver on a node? that is when it only has calico-node ds rolled out on it?).
Or will there be a new Helm chart released soon to have this v1.28.11 for tigera operator?
Many thanks!

[tmjd]

I would suggest updating to v3.25 rather than overriding the image. I don't know when or if there will be an update to v3.24 to include v1.28.11.

I believe you can disable CSI-Node-Driver by setting kubeletVolumePluginPath: None on the Installation resource. I believe it is only necessary for Envoy and Istio. @Josh-Tigera can you confirm that?

Since this has been fixed in more recent versions by now having the tolerations for csi-node-driver matching calico-node and also it is possible to override the tolerations I'm going to close this issue.