Support dynamic provisioning for CSI migration scenarios #73653

ddebroy · 2019-02-02T05:15:50Z

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test

/kind feature

/kind flake

What this PR does / why we need it:
This PR introduces a set of changes needed in in-tree PV Controller to call out to the external provisioning controller for in-tree plugins if they are migratable to CSI. The basic steps are:

Check if in-tree plugin is migratable to CSI. If so:
Set annStorageProvisioner in the PVC to CSI plugin name that superseded the specified in-tree plugin.
Do not proceed with in-tree provisioning and return so that external provisioner can act on the claim.

In addition to the above, the PR also introduces a new API TranslateInTreeStorageClassParametersToCSI in the CSI translation staging repo. This will be used by the external provisioner to translate from in-tree storage class parameters to CSI plugin parameters.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:
Part of enhancements for kubernetes/enhancements#625

For the changes in this PR to take an effect, we need a separate set of changes in https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner.git which does the following:

Enhances getStorageClassFields to check (through GetCSINameFromIntreeName) if the storageclass provisioner has a corresponding CSI plugin that supersedes it. If so:
Makes getStorageClassFields aware that it's handling dynamic volume provisioning for an in-tree plugin that has been migrated to CSI because [a] For regular in-tree provisioning path, external provisioning will never get invoked [b] For regular CSI provisioning path, GetCSINameFromIntreeName would not return anything for a CSI plugin name.
Enhances getStorageClassFields to [a] translate in-tree storage class parameters to the corresponding CSI plugin parameters using TranslateInTreeStorageClassParametersToCSI [b] return the CSI plugin name (instead of the storage class in-tree provisioner) as the provisioner name (so that it aligns with ctrl.provisionerName) and [c] return a boolean to provisionClaimOperation indicating dynamic provisioning in a CSI migration context
Enhances provisionClaimOperation to invoke TranslateCSIPVToInTree for dynamic provisioning in a CSI migration context.

The above PR depends on TranslateInTreeStorageClassParametersToCSI from this PR to be staged first.

Does this PR introduce a user-facing change?:

Introduce dynamic volume provisioning shim for CSI migration

/sig-storage
/assign @jsafrane @saad-ali
/cc @msau42 @leakingtapan @davidz627

ddebroy · 2019-02-02T23:09:36Z

/sig storage

davidz627 · 2019-02-04T23:21:06Z

@ddebroy thanks for this PR! I will review shortly.

Would recommend that you create a separate PR for the translation library staging change. Since the external provisioner depends on it lets fast track that and merge it separately.

dims · 2019-02-05T00:25:00Z

pkg/controller/volume/persistentvolume/pv_controller.go

+			// Set provisionerName to CSI plugin name for setClaimProvisioner
+			provisionerName, err = csitranslation.GetCSINameFromIntreeName(storageClass.Provisioner)
+			if err != nil {
+				strerr := fmt.Sprintf("Failed to get CSI name for In tree plugin: %s", storageClass.Provisioner)


can we print the err so it's in strerr itself?

also we can drop the klog as we are returning the err as well as logging it.

The convention in provisionClaimOperation appears to be to klog.V(2|3).Infof(...) all errors besides returning them as well as reporting the event with ctrl.eventRecorder.Event. So I was following the convention elsewhere in provisionClaimOperation. @jsafrane please confirm the convention.

I will change the error message to include the whole error as suggested.

ddebroy · 2019-02-05T08:11:24Z

PR #73734 supersedes the csi-translation-lib bits of of this PR.

jsafrane · 2019-02-05T13:51:13Z

pkg/controller/volume/persistentvolume/pv_controller.go

 	if plugin != nil {
-		pluginName = plugin.GetPluginName()
+		if plugin.IsMigratedToCSI() {


Please add an unit test for this. "11-17 - external provisioner" could be a good start. You need to inject a volume plugin that's migrated to CSI, some variant of wrapTestWithPluginCalls might be sufficient.

Updated PR with 11-21 - external provisioner for CSI migration testcase to make sure for CSI migration scenarios, provisioning ends up waiting for a claim with annotation vendor.com/MockCSIPlugin and event Normal ExternalProvisioning.

jsafrane · 2019-02-05T13:52:46Z

staging/src/k8s.io/csi-translation-lib/plugins/aws_ebs.go

@@ -40,6 +40,11 @@ func NewAWSElasticBlockStoreCSITranslator() InTreePlugin {
 	return &awsElasticBlockStoreCSITranslator{}
 }

+// TranslateInTreeStorageClassParametersToCSI translates InTree EBS storage class parameters to CSI storage class


IMO, this can be removed now.

jsafrane

Thanks for the test! Please remove one commented-out line and I think we're good to go.

jsafrane · 2019-02-26T13:03:59Z

pkg/controller/volume/persistentvolume/pv_controller.go

+
+	// For testing only: hook to intercept CSI driver name <=> Intree plugin name mapping
+	// Not used when set to nil
+	csiNameFromIntreeNameHook func(pluginName string) (string, error)


In long term it would be better if CSI translation library provided an interface instead of function calls so its callers can create fake implementations for unit tests. As a separate PR, of course, no need to fix it here.

xref" #74594

jsafrane · 2019-02-26T13:04:40Z

pkg/controller/volume/persistentvolume/pv_controller.go

+			// of not setting pluginName for external provisioners (including CSI)
+			// Set provisionerName to CSI plugin name for setClaimProvisioner
+			provisionerName, err = ctrl.getCSINameFromIntreeName(storageClass.Provisioner)
+			// provisionerName, err = csitranslation.GetCSINameFromIntreeName(storageClass.Provisioner)


nit: please remove

Signed-off-by: Deep Debroy <ddebroy@docker.com>

ddebroy · 2019-02-26T16:43:05Z

Addressed code review comment and squashed commits

davidz627 · 2019-02-26T22:16:30Z

/lgtm

jsafrane · 2019-02-27T18:09:25Z

/approve

k8s-ci-robot · 2019-02-27T18:09:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ddebroy, jsafrane

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/controller/volume/persistentvolume/OWNERS~~ [jsafrane]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fejta-bot · 2019-02-27T21:38:10Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2019-02-28T04:38:10Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2019-02-28T07:47:10Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

k8s-ci-robot assigned jsafrane Feb 2, 2019

k8s-ci-robot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Feb 2, 2019

k8s-ci-robot assigned saad-ali Feb 2, 2019

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Feb 2, 2019

k8s-ci-robot requested review from davidz627, leakingtapan and msau42 February 2, 2019 05:15

ddebroy changed the title ~~Provisioning changes for CSI migration~~ Support dynamic provisioning for CSI migration scenarios Feb 2, 2019

k8s-ci-robot added the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Feb 2, 2019

dims reviewed Feb 5, 2019

View reviewed changes

ddebroy mentioned this pull request Feb 5, 2019

Add CSI Translation API for Storage Class parameters #73734

Merged

jsafrane reviewed Feb 5, 2019

View reviewed changes

ddebroy mentioned this pull request Feb 9, 2019

Support dynamic provisioning for CSI migration scenarios kubernetes-sigs/sig-storage-lib-external-provisioner#28

Closed

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 17, 2019

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Feb 25, 2019

jsafrane reviewed Feb 26, 2019

View reviewed changes

PV Controller changes to support provisioning in CSI Migration scenarios

23478f1

Signed-off-by: Deep Debroy <ddebroy@docker.com>

k8s-ci-robot assigned davidz627 Feb 26, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 26, 2019

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 27, 2019

k8s-ci-robot merged commit 3678704 into kubernetes:master Feb 28, 2019

ddebroy mentioned this pull request Mar 1, 2019

Support additional provisioner names for CSI migration scenarios kubernetes-sigs/sig-storage-lib-external-provisioner#34

Merged

ddebroy mentioned this pull request Apr 15, 2019

External provisioner migration notes kubernetes-csi/external-provisioner#190

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support dynamic provisioning for CSI migration scenarios #73653

Support dynamic provisioning for CSI migration scenarios #73653

ddebroy commented Feb 2, 2019

ddebroy commented Feb 2, 2019

davidz627 commented Feb 4, 2019

dims Feb 5, 2019

ddebroy Feb 7, 2019

ddebroy commented Feb 5, 2019 •

edited

Loading

jsafrane Feb 5, 2019

ddebroy Feb 25, 2019

jsafrane Feb 5, 2019

jsafrane left a comment

jsafrane Feb 26, 2019

ddebroy Feb 26, 2019

davidz627 Feb 26, 2019

jsafrane Feb 26, 2019

ddebroy Feb 26, 2019

ddebroy commented Feb 26, 2019

davidz627 commented Feb 26, 2019

jsafrane commented Feb 27, 2019

k8s-ci-robot commented Feb 27, 2019

fejta-bot commented Feb 27, 2019

fejta-bot commented Feb 28, 2019

fejta-bot commented Feb 28, 2019

Support dynamic provisioning for CSI migration scenarios #73653

Support dynamic provisioning for CSI migration scenarios #73653

Conversation

ddebroy commented Feb 2, 2019

ddebroy commented Feb 2, 2019

davidz627 commented Feb 4, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ddebroy commented Feb 5, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsafrane left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ddebroy commented Feb 26, 2019

davidz627 commented Feb 26, 2019

jsafrane commented Feb 27, 2019

k8s-ci-robot commented Feb 27, 2019

fejta-bot commented Feb 27, 2019

fejta-bot commented Feb 28, 2019

fejta-bot commented Feb 28, 2019

ddebroy commented Feb 5, 2019 •

edited

Loading