Improve CSI Plugin compatibility docs

[nvx]

Background

Currently the page https://www.nomadproject.io/docs/internals/plugins/csi#csi-plugins links to the Kubernetes CSI plugin list with the text Any of these plugins should work with Nomad out of the box.. Earlier it also mentions some example CSI plugins by name such as vSphere.

Unfortunately in practice, it seems some CSI plugins go outside the CSI spec and also make calls to the Kubernetes API which makes using these plugins outside of Kubernetes impossible (short of implementing a Kubernetes compatible API interface anyway). This includes the vSphere CSI plugin mentioned on the page as per kubernetes-sigs/vsphere-csi-driver#542 and a quick Google indicates this might not be the only plugin doing such a thing, for example OpenEBS gets a mention here https://stackoverflow.com/a/52419063/1276877

While I'm not sure how legitimate it is for CSI plugins to do this sort of thing or how commonplace it is in the Kubernetes ecosystem, the current state means that there's no easy way to tell if a plugin will work with Nomad.

Another possible consideration is some CSI plugins may rely on CSI APIs not yet implemented by Nomad (I see a few open feature requests for some APIs) but I'm not sure how much of an issue this is in practice (ie, if it just is needed for extra functionality vs required to use a given CSI plugin entirely).

Proposal

While the Kubernetes CSI plugin list seems to be the most authoritative source, it naturally doesn't consider non-Kubernetes uses. As such maintaining a list of CSI plugins in the Nomad docs that are known to either be compatible or incompatible with Nomad would be an incredibly useful reference for Nomad operators.

This visibility could also help pressure implementers of CSI plugins to not deviate outside of the CSI spec and maybe cast light on any deficiencies in functionality that could be addressed in future versions of the CSI spec.

[lgfa29]

Thank you for the idea @nvx.

It is true that, unfortunately, some CSI plugins don't follow the spec 100% and end up relying on Kubernetes-specific APIs. Nomad support for CSI is also not complete, so this creates a challenging situation.

We are collecting some working CSI examples in the demo/csi folder of our repo. We could mention this folder in our docs as well.

[magikid]

One I see that is missing from the demo/csi folder is democratic-csi

[tgross]

Hi @magikid! We're actually in the process of creating a set of Nomad Packs in the community registry for a handful of plugins. Included in those is democratic-csi: hashicorp/nomad-pack-community-registry#113