Add block volume support to internal provisioners.

[jsafrane]

What this PR does / why we need it:
Internal provisioners now create filesystem PVs when block PVs are requested. This leads to unbindable PVCs.

In this PR, volume plugins that support block volumes provision block PVs when block is requested. All the other provisioners return clear error in kubectl describe pvc:

Events:
  Type     Reason              Age               From                         Message
  ----     ------              ----              ----                         -------
  Warning  ProvisioningFailed  7s (x2 over 18s)  persistentvolume-controller  Failed to provision volume with StorageClass "standard": kubernetes.io/cinder does not support block volume provisioning

AWS EBS, Azure Disk, GCE PD and Ceph RBD volume plugins support dynamic provisioning of raw block volumes.

cc @kubernetes/vmware for vsphere changes
cc @andyzhangx for Azure changes

/assign @copejon @mtanino

[jsafrane]

e2e tests will follow

[andyzhangx]

/lgtm
on azuredisk & azurefile part

[jsafrane]

Thanks for the quick review, however,
/lgtm cancel

I need the others reviewer too :-)

[jsafrane]

/approve
stupid bot!

[jsafrane]

Added e2e test. I ran it manually on GCE and AWS. It will be run in alpha test suite(s), since it depends on alpha feature.

[childsb]

This seems like a feature, but its really a bug fix:

1. we added block volume support for SOME of the plugins
2. The plugins which we didn't explicitly get block volume support don't know that they didn't
3. the provisioners for the volume types which dont have block volume support still provision regular volumes when the block volume flag is set in the storage class.

ideally these volume types wouldn't even provision volumes, but enforcing that may be more difficult then enabling them to provision block volumes.

would like this in as a fix for 1.11

[jsafrane]

/kind bug
/priority critical-urgent

[humblec]

"lgtm" from gluster side, Thanks Jan!

[erinboyd]

@mkimuram @screeley44 @msau42 please review

[jsafrane]

/retest

[mkimuram]

I think that it's enough to have a check in Provision() call in each volume plugin. They're responsible to provision something that corresponds to given PVC, incl. block volume support.

I agree with you that it is enough for checking current internal provisioner.

Traditionally, we leave this to users / cluster admins. It's their responsibility to run their clusters with compatible components, i.e. with the provisioners that support block. And that's what are alpha/beta feature gates good for. I know it's not ideal, however, there is no direct call-like interface between Kubernetes and external provisioners.

However, IIUC, we are on the way to migrate provision logic for all plugins to CSI,
and CSI provisioner will not have a similar place to check whether each CSI driver supports block volume, because they share the same Provision() and call CreateVolume() for each driver inside it(*1)(*2).

If we don't check it, we won't be able to make filebase backend, like NFS and file mode of Gluster/Ceph, co-exist in blockVolume-enabled environment.
(Assuming that filebase backend can't support blockVolume, and blockVolume-enabled environment only allows blockVolume supported drivers without this kind of check.)

So, I think that we also need to find a way to check it in external provisioner.
(It doesn't need to be the same logic to internal provisioner, so we can separate this issue from this PR, though.)

(*1) provisionClaimOperation() for external provisioner.
https://github.com/kubernetes-incubator/external-storage/blob/master/lib/controller/controller.go#L887
(*2) Provision() call for CSI
https://github.com/kubernetes-csi/external-provisioner/blob/master/pkg/controller/controller.go#L280

[jsafrane]

For CSI we have external provisioner that will get updated with a particular Kubernetes release and will check what PVC wants and if the CSI driver supports it.

[msau42]

For CSI we have external provisioner that will get updated with a particular Kubernetes release

Users could run older versions of the CSI driver though so we still need to handle the scenario that the external provisioner may ignore any new provisioning parameters

[screeley44]

aws and e2e LGTM

[mkimuram]

To discuss similar check for external provisioner, I made PR for it.

• [WIP] Add block volume support check to external provisioners
  [WIP] Add block volume support check to external provisioners kubernetes-retired/external-storage#787

As described there, similar issue to CSI also happens in flex volume plugin.

[rootfs]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx, jsafrane, rootfs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/volume/OWNERS [jsafrane,rootfs]
• test/e2e/storage/OWNERS [jsafrane,rootfs]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jsafrane]

I got reviews from all volume plugins (@rootfs for Cinder and RBD)

/sig storage

[jsafrane]

Action required: This pull request must have the status/approved-for-milestone label applied by a SIG maintainer.

@childsb @saad-ali, can you please make sure this is merged in 1.11?

[dims]

/status approved-for-milestone

[k8s-github-robot]

[MILESTONENOTIFIER] Milestone Pull Request: Up-to-date for process

@andyzhangx @copejon @jsafrane @mtanino @rootfs

Pull Request Labels

• sig/storage: Pull Request will be escalated to these SIGs if needed.
• priority/critical-urgent: Never automatically move pull request out of a release milestone; continually escalate to contributor and SIG through all available channels.
• kind/bug: Fixes a bug discovered during the current release.

Help

• Additional instructions
• Commands for setting labels

[k8s-github-robot]

/test all [submit-queue is verifying that this PR is safe to merge]

[k8s-ci-robot]

@jsafrane: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-integration	408f62c	link	/test pull-kubernetes-integration
pull-kubernetes-e2e-gce	408f62c	link	/test pull-kubernetes-e2e-gce

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 63348, 63839, 63143, 64447, 64567). If you want to cherry-pick this change to another branch, please follow the instructions here.