Add Container Storage Interface integration to conformance

[msau42]

What would you like to be added:
Continuing the discussion in #65155 (comment), while we can't test provider-specific CSI drivers in conformance, we can test provider-agnostic drivers like CSI hostpath, and make sure that K8s distributions are properly supporting the CSI interface.

There are some challenges that need to be solved before we can test CSI in conformance:

• sig-storage has refactored some 30+ tests to be driver-agnostic (can run against any driver, in-tree or CSI). However, that currently makes it difficult to promote those tests to conformance using the current conformance tooling because the tests case definitions are now a library shared by many drivers. We either need to: 1) enhance the current conformance tooling to handle conditional test cases based off of other inputs (such as the driver being tested) or 2) duplicate all of the test cases to a conformance-specific suite.
• CSI drivers need to be privileged in order to do mount propagation on kubelet. I forget if conformance tests allow privileged containers?
• CSI hostpath needs "/dev" access on the host to support raw block (filesystem PVC is fine, the driver just writes to the container fs). It may be that we can't test raw block in conformance.
• In addition, we have some test cases outside of our driver suite that uses the mock driver, and checks various CSI calls toggled by different settings (via K8s objects like CSIDriver, as well as CSI driver config options). Most of the tests validate that the right CSI calls are made by grepping mock driver logs for specific msgs since the mock driver isn't actually very functional (compared to the hostpath driver). Right now most of these test cases are testing beta features, but it would be a good idea to write a very basic test testing all the GA features. These test cases may be less functional than the generic driver suite since the mock driver doesn't actually write any data and most of our functional tests check for data persistence.

Why is this needed:
It is important to make sure that all K8s distributions support the CSI spec.

@kubernetes/sig-storage-feature-requests
cc @johnbelamaric

[msau42]

/assign

[johnbelamaric]

@brahmaroutu is working on a KEP for this

[johnbelamaric]

Privileged is not ideal but other things also need privileged too. Eventually maybe we will be able to segregate those but we cannot now.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[msau42]

/lifecycle frozen

[msau42]

@huffmanca would you be able to look into promoting the skip attach and pod info csi mock volume tests to conformance now that the features are GA?

[huffmanca]

I can look into this!

[huffmanca]

/assign

[msau42]

Note our csi mock tests are dependent on kubectl exec which is not part of conformance.

Also, we plan on adding a proxy mode for csi mock driver, which would add a dependency on port forwarding, which is also not part of conformance.

[pohly]

Another technical issue: some clusters may use a non-standard kubelet data directory, in which case the current test driver deployments all fail because kubelet doesn't find the driver. We need a parameter for that and then someone running conformance tests must set that parameter if necessary - tracked in #92664

[pohly]

https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md#conformance-test-requirements also specifies that a test doesn't need privileges to run. We'll have to disable all mount operations when running tests.

[msau42]

I spoke to @johnbelamaric earlier about the privileged requirement and concluded that we can relax it if necessary. So I don't consider requiring privileged an issue.