Promote some InTreePluginXXXUnregister to GA

[carlory]

What type of PR is this?

/kind cleanup
/sig storage

What this PR does / why we need it:

The following plugins in the Special notes for your reviewer section already completed the migration task and its in-tree code was removed as part of the migration process. So, the corresponding feature gate should be removed, but it has not been removed yet.

Whatever the value of the feature gate is, it won't affect the code's behavior. Even if the feature gate is false, we will still get the right value from csinode's annotations. The PR updates the value to true and promotes them to GA. And those feature gates will be removed in v1.32.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

KEPs:

• AWS EBS in-tree to CSI driver migration enhancements#1487
• Azure disk in-tree to CSI driver migration enhancements#1490
• Azure file in-tree to CSI driver migration enhancements#1885
• GCE PD in-tree to CSI driver migration enhancements#1488
• Cinder in-tree to CSI driver migration enhancements#1489
• vSphere in-tree to CSI driver migration enhancements#1491

/cc @leakingtapan @andyzhangx @davidz627 @adisky @dims @divyenpatel @xing-yang

Does this PR introduce a user-facing change?

Promoted the following feature gates to general availability:
- `InTreePluginAWSUnregister`
- `InTreePluginAzureDiskUnregister`
- `InTreePluginAzureFileUnregister`
- `InTreePluginGCEUnregister`
- `InTreePluginOpenStackUnregister`
- `InTreePluginvSphereUnregister`

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[k8s-ci-robot]

@carlory: GitHub didn't allow me to request PR reviews from the following users: davidz627.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

What type of PR is this?

/kind cleanup
/sig storage

What this PR does / why we need it:

The following plugins in the Special notes for your reviewer section already completed the migration task and its in-tree code was removed as part of the migration process. So, the corresponding feature-gate should be removed as well, but it is not removed yet.

Whatever the value of the feature-gate is, it won't have any effect on the behavior of the code. Even if the feature-gate is set to false, we will still get the right value from csinode's annotations. The PR is to promote their to GA. And those feature-gates will be removed in v1.32.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

KEPs:

• AWS EBS in-tree to CSI driver migration enhancements#1487
• Azure disk in-tree to CSI driver migration enhancements#1490
• Azure file in-tree to CSI driver migration enhancements#1885
• GCE PD in-tree to CSI driver migration enhancements#1488
• Cinder in-tree to CSI driver migration enhancements#1489
• vSphere in-tree to CSI driver migration enhancements#1491

/cc @leakingtapan @andyzhangx @davidz627 @adisky @dims @divyenpatel @xing-yang

Does this PR introduce a user-facing change?

Promote the following feature-gates to GA:
- InTreePluginAWSUnregister
- InTreePluginAzureDiskUnregister
- InTreePluginAzureFileUnregister
- InTreePluginGCEUnregister
- InTreePluginOpenStackUnregister
- InTreePluginvSphereUnregister

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[carlory]

/test pull-kubernetes-unit

[carlory]

/cc @jsafrane

[xing-yang]

I don't think we should move a featuregate directly from "Alpha" to "GA".

cc @jsafrane @msau42

[carlory]

but the in-tree code was removed, so the beta stage may be meaningless.

[xing-yang]

/hold

Putting this on hold until we understand the implications of moving this to GA. If this means the user cannot use in-tree PVs/PVCs/StorageClasses any more, then we can never enable this by default.

cc @jsafrane

[carlory]

FYI: https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/625-csi-migration/README.md#rollout-upgrade-and-rollback-planning

[jsafrane]

Thanks for the enhancement link! I checked the PR and tested with in-tree AWS on Kubernetes 1.29, CSI migration works fine. We should have declare these feature gates GA when CSI migration went GA and remove them when CSI migration gates were removed.

What if we just delete the feature gates? They don't affect anything, you can turn them on and off, but they don't have any impact on any code in Kubernetes.

For example, InTreePluginAWSUnregister is used only in IsMigrationCompleteForPlugin(), which is used only by A/D controller to see if there could be a node that still has kubelet with CSI migration enabled disabled. There can be no such kubelet, AWS CSI migration went GA in 1.24 and the oldest kubelet that 1.31 KCM supports is 1.27. And even if it was such a kubelet, A/D controller does not have the in-tree volume plugin to attach/detach its non-migrated volumes anyway, we removed the volume plugin in 1.27.

@carlory can you please split it into two PRs, one that updates all the unit tests (most of this PR) and a second one that deletes the feature gates and all their usage? That one should be small and we can argue if it's safe or not to remove them.

[xing-yang]

We'll need to test this on vSphere before this change is merged.

[carlory]

FYI:

• CSIMigrationvSphere is GA in 1.27. Lock CSIMigrationvSphere feature gate for k8s 1.27 #116610.
• the CSIMigrationvSphere feature-gate was removed in 1.29. Remove GA featuregate about CSIMigrationvSphere in 1.29 #121291
• in-tree code was removed in 1.30. remove the deprecated in-tree vsphere volume's code #122090.

So 1.29 is the last release that can handle version skew between managers and nodes. It's not related to this PR.

[carlory]

https://github.com/kubernetes/kubernetes/blob/227c2e7c2b2c05a9c8b2885460e28e4da25cf558/pkg/controller/volume/attachdetach/util/util.go#L271-L332

[carlory]

According to the https://kubernetes.io/releases/version-skew-policy/, kube-apiserver is at 1.29 and kubelet is supported at 1.29, 1.28, 1.27, and 1.26.

So if kube-apiserver is at 1.30 and kubelet is supported at 1.30, 1.29, 1.28, and 1.27.

[carlory]

the CSIMigrationvSphere feature-gate was GA at 1.27, so csinode's annotation storage.alpha.kubernetes.io/migrated-plugins always contains the kubernetes.io/vsphere-volume item.

[carlory]

In other words, operations for the in-tree vsphereVolume type are redirected to the csi.vsphere.vmware.com CSI driver.

[xing-yang]

cc @divyenpatel

[carlory]

can you please split it into two PRs, one that updates all the unit tests (most of this PR) and a second one that deletes the feature gates and all their usage? That one should be small and we can argue if it's safe or not to remove them.

I will do it

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: carlory
Once this PR has been reviewed and has the lgtm label, please assign xing-yang for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• pkg/features/OWNERS
• pkg/volume/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[pacoxu]

/test pull-kubernetes-e2e-gce-cos-alpha-features

[pacoxu]

You can change https://github.com/kubernetes/test-infra/blob/ae8f22151440e45c688ca6fdea54fbea50ab9e14/config/jobs/kubernetes/sig-cloud-provider/gcp/gcp-gce.yaml#L458 to remove the FG setting there.

[sftim]

Changelog suggestion

Promoted the following feature gates to general availability:
- `InTreePluginAWSUnregister`
- `InTreePluginAzureDiskUnregister`
- `InTreePluginAzureFileUnregister`
- `InTreePluginGCEUnregister`
- `InTreePluginOpenStackUnregister`
- `InTreePluginvSphereUnregister`

[dims]

/test all

[carlory]

replaced by #124815

[carlory]

/reopen

[k8s-ci-robot]

@carlory: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

@carlory: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubernetes-unit	eb52ea8	link	true	/test pull-kubernetes-unit

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.