Add Block volume support for CSI provisioner

[mkimuram]

CSI provisioner should provision Block volumes properly.

In CSI provisioner, below three logics will need to be implemented, to add Block volume support to CSI provisioner:

1. Add SupportsBlock that properly returns whether Storage Provider's plugin supports block (this will be checked by using ValidateVolumeCapabilities),
2. Pass BlockVolume instead of MountVolume to CreateVolume if volumeMode is set to be Block on Provision,
3. Set volumeMode to PV returned by Provision.

In Storage Provider's plugin, below two logics should be implemented:

1. Return true to ValidateVolumeCapabilities with BlockVolume specified if it supports block, and return false if it doesn't,
2. Properly handle CreateVolume with BlockVolume specified, if ValidateVolumeCapabilities with BlockVolume returns true.
   (In most case, no special handling would be needed, as far as I see existing provisioner/attacher codes.)

[mkimuram]

@wongma7

Now kubernetes-retired/external-storage#830 is merged, we are ready to implement BlockProvisioner interface for CSi provisioner. I will implement prototype of codes with above logic and will ask you for review.

[mkimuram]

@wongma7

I've implemented a prototype and confirmed that it works well with CSI RBD driver. PTAL
I will write unit tests and make a PR, if implementation policy looks good to you.

In addition, as a result of my investigation on the current CSI driver implementations, all the drivers below use blacklisting for ValidateVolumeCapabilities and they don't check AccessType and just checks AccessMode, currently. (Portworx checks whether Mount or Block is specified, though.)
Therefore, this kind of CSI drivers which really support Block should work without modification for CSI driver's side, because their ValidateVolumeCapabilities will return true for Block. On the other hand, CSI drivers which don't support Block, like Cpeh, will need to add logic to return false when AccessType is Block.

• CSI drivers

Driver	Block Support	ValidateVolumeCapabilities for block	Need modification
DigitalOcean Block Storage	Yes?	Return true	No
GCE	Yes	Return true	No
Portworx	Yes?	Return true	No
RBD	Yes	Return true	No
Ceph	No	Return true	Yes
ScaleIO	Yes?	Return true	No
vSphere	Yes?	Return true	No

For Cinder, OpenSDS and NetApp, I guess that they need to return proper value for ValidateVolumeCapabilities to their background storage, because they will handle multiple backend storages.

With above situation in mind, we might need to consider when to merge this feature, (a) Merge this feature after all public CSI drivers are confirmed to handle block properly, or (b) Merge this feature first and wait for all CSI drivers handle it properly after that. I think that (b) is O.K. as long as Block feature is default to false. But once it become default to true (and we plan to make it beta in this release), all drivers should be able to handle it, properly.

[mkimuram]

Also added unit test cases. Please review them.

[mkimuram]

On the other hand, CSI drivers which don't support Block, like Cpeh, will need to add logic to return false when AccessType is Block.

Above is fixed in issue.

For Cinder, OpenSDS and NetApp, I guess that they need to return proper value for ValidateVolumeCapabilities to their background storage, because they will handle multiple backend storages.

For multiple backend storages, I will investigate more.
Could you give us any advices for it from OpenSDS viewpoint? @xing-yang

[wongma7]

I added one small comment to your proposed changes.

(b) sounds okay to me too but I am totally ignorant about how we expect CSI plugins to be maintained in relation to kube/other COs releases

[mkimuram]

@wongma7

I added one small comment to your proposed changes.

Thank you for your review and comment.
I've fixed Provision not to set fsType if it is Block PV and made PR. PTAL

Also, I updated under vendor directory to include kubernetes-retired/external-storage#830 , instead of manually modifying it.
@rootfs Thank you for your advice in doing it. I would appreciate it if you could also check the results.

[mkimuram]

@wongma7

I made below PR to add CSI rbd/Cephfs e2e test:
kubernetes/kubernetes#67088
Also, I made below PR to make volume mode e2e test executed in a general way:
kubernetes/kubernetes#66577

Once both of them are merged, we are ready to add block volume e2e test for CSI drivers to test this feature in e2e.

[xing-yang]

@mkimuram Sorry for the late response. I just noticed your question about OpenSDS. Currently OpenSDS only has block API support. This means OpenSDS CSI driver should be able to support block volume mode for all the backends it supports, but we have not got chance to validate the block volume mode support yet. I believe Cinder is in the same situation because it only supports block APIs.

When we add File API support in OpenSDS, things will change and backends supported through File API should be only supporting filesystem volume mode.

[mkimuram]

@xing-yang

Thank you for your comment on OpenSDS and Cinder.

I understand that currently OpenSDS and Cinder support block only, so they will be able to always return true to ValidateVolumeCapabilities with BlockVolume for any backends.
Also, even after OpenSDS supports File API, it will be a separate API, so File API will be able to always return false to ValidateVolumeCapabilities with BlockVolume for any backends.

Therefore, they should be able to work well with current implementation.

[xing-yang]

@mkimuram You are right!

[msau42]

cc @lpabon @saad-ali @vladimirvivien

I think this issue covers the block issues Luis was talking about this morning