KEP-2924: Enable CSI migration for cephfs intree plugin

keps/prod-readiness/sig-storage/2924.yaml

@@ -0,0 +1,3 @@
+kep-number: 2924
+alpha:
+  approver: "@wojtek-t"

keps/sig-storage/2924-csi-migration-cephfs/README.md

@@ -0,0 +1,52 @@
+# In-tree Storage Plugin to CSI Migration - Ceph Cephfs Design Doc
+
+## Table of Contents
+
+<!-- toc -->
+- [Summary](#summary)
+  - [New Feature Gates](#new-feature-gates)
+- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
+- [Implementation History](#implementation-history)
+<!-- /toc -->
+
+
+## Summary
+
+This document present as a vendor specific KEP for the parent KEP
+[CSI Migration](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/625-csi-migration)
+
+This inherits all the contents from its parent KEP. It will introduce two new feature gates to be 
+used as described in its parent KEP. For all other contents, please refer to the parent KEP.
+
+### New Feature Gates
+
+- CSIMigrationCephfs
+  - As describe in [CSI Migration](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/625-csi-migration), 
+  when this feature flag && the `CSIMigration` is enabled at the same time, all operations related to the 
+  in-tree volume plugin `kubernetes.io/cephfs` will be redirected to use the corresponding CSI driver. From a
+  user perspective, nothing will be noticed.
+- InTreePluginCephFSUnregister
+  - This flag technically is not part of CSI Migration design. But it happens to be related and helps with 
+  CSI Migration. The name speaks for itself, when this flag is enabled, kubernetes will not register the 
+  `kubernetes.io/cephfs` as one of the in-tree storage plugin provisioners. This flag standalone can work out 
+  of CSI Migration features.
+  - However, when all `InTreePluginCephfsUnregister`, `CSIMigrationCephfs` and `CSIMigration` feature 
+  flags are enabled at the same time. The kube-controller-manager will skip the feature flag checking 
+  on kubelet and treat Ceph Cephfs CSI migration as already complete. And directly redirect traffic to CSI 
+  driver for all cephfs related operations.
+
+
+## Production Readiness Review Questionnaire
+
+Please refer to the [CSI Migration Production Readiness Review Questionnaire](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/625-csi-migration#production-readiness-review-questionnaire).
+
+## Implementation History
+
+Major milestones in the life cycle of a KEP should be tracked in `Implementation History`.
+
+- 2022-01-27 KEP created
+
+Major milestones for Ceph Cephfs in-tree plugin CSI migration:
+
+- 1.24
+  - Ceph Cephfs CSI migration to Alpha

keps/sig-storage/2924-csi-migration-cephfs/kep.yaml

@@ -0,0 +1,53 @@
+title: In-tree Storage Plugin to CSI Migration - Ceph Cephfs
+kep-number: 2924
+authors:
+  - "@humblec"
+owning-sig: sig-storage
+participating-sigs:
+  - sig-cluster-lifecycle
+reviewers:
+  - "@Jiawei0227"
+  - "@msau42"
+approvers:
+  - "@msau42"
+editor: "@Jiawei0227"
+creation-date: 2022-01-27
+last-updated: 2022-01-27
+disable-supported: true
+status: implementable
+see-also:
+  - "https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-migration.md"
+prr-approvers:
+  - "@wojtek-t"
+replaces:
+
+# The target maturity stage in the current dev cycle for this KEP.
+stage: alpha
+
+# The most recent milestone for which work toward delivery of this KEP has been
+# done. This can be the current (upcoming) milestone, if it is being actively
+# worked on.
+latest-milestone: "v1.24"
+
+# The milestone at which this feature was, or is targeted to be, at each stage.
+milestone:
+  alpha: "v1.24"
+
+# The following PRR answers are required at alpha release
+# List the feature gate name and the components for which it must be enabled
+feature-gates:
+  - name: CSIMigrationCephfs
+    components:
+    - kube-controller-manager
+    - kubelet
+    - kube-scheduler
+  - name: InTreePluginCephfsUnregister
+    components:
+    - kube-controller-manager
+    - kubelet
+    - kube-scheduler
+
+# The following PRR answers are required at beta release
+metrics:
+  - csi_sidecar_duration_operation
+  - storage_operation_duration_seconds