CSI controller crashes in deployments using current templates

[laozc]

/kind bug

What steps did you take and what happened:
CSI plugin needs insecure-flag to be set without trusted CA even when the thumbprint is provided.
The templates in the CAPV source does not contain this flag.

What did you expect to happen:

Anything else you would like to add:
CAPV does not require the insecure flag when the thumbprint is provided.
This should be aligned in the future to avoid customer confusion.

Environment:

• Cluster-api-provider-vsphere version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[sbueringer]

Hm that's not great.

@chrischdi Didn't we use our normal templates for our tests with that combination of config?

[chrischdi]

I thought we do not need the insecure flag when there is a thumbprint: https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/blob/main/templates/cluster-template.yaml#L451

This was actually introduced to be like that in #1819 .

[sbueringer]

I think we also used this cluster-template like this for our CAPI scale test where we setup a CAPV cluster?
(I'm not exactly sure if we validated the whole thing including that CSI is up/working)

[laozc]

We may grab the config from the cluster to double check.
The image in my provisioned cluster is gcr.io/cloud-provider-vsphere/csi/release/driver:v2.1.0 and it keeps crashing without the flag.

[chrischdi]

I think I did see the same issue while running locally yesterday (did not have time though to invest).

Wondering how the conformance tests work (or if they test pvc's).

[sbueringer]

Probably don't test PVC's because they can't assume a CSI is deployed

[sbueringer]

Maybe the old CSI driver version could explain it? IIRC current version is 3 something? I think we should upgrade that in any case and maybe that also solves the problem?

[laozc]

Tried to bump the CSI driver to latest 3.1.0 and it solved the issue.