Changes to snapshot controller to add sourceVolumeMode

[RaunakShah]

What type of PR is this?
/kind feature

What this PR does / why we need it:

This PR adds changes to snapshot-controller to read the volume mode of the source PV and add it to the Spec.SourceVolumeMode field of the to-be-created VolumeSnapshotContent object.

Also introduce a feature flag for this feature.

KEP - https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3141-prevent-volume-mode-conversion

Testing

1. Deploy Hostpath CSI driver with v6.0.0-rc2 version of csi-snapshotter and VolumeSnapshotContent CRD. Set prevent-volume-mode-conversion to true in snapshot-controller.

% kubectl get deployment -n kube-system -o yaml snapshot-controller
apiVersion: apps/v1
kind: Deployment
...
spec:
  minReadySeconds: 15
  progressDeadlineSeconds: 600
...
spec:
      containers:
      - args:
        - --v=5
        - --leader-election=true
        - --prevent-volume-mode-conversion=true
        image: snapshot-controller:latest
...

2. Create a VolumeSnapshot object from an existing PVC:

% kubectl get volumesnapshot -A
NAMESPACE   NAME                   READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS               SNAPSHOTCONTENT                                    CREATIONTIME   AGE
default     new-snapshot-demo-v1   true         hpvc                                1Gi           csi-hostpath-snapclass-v1   snapcontent-f4161ad9-9279-4e77-93ee-f8b5898b6010   4d             4d

3. Verify VolumeSnapshotContent object for SourceVolumeMode field:

% kubectl get volumesnapshotcontent -o yaml
apiVersion: v1
items:
- apiVersion: snapshot.storage.k8s.io/v1
  kind: VolumeSnapshotContent
...
  spec:
    deletionPolicy: Delete
    driver: hostpath.csi.k8s.io
    source:
      volumeHandle: 37a4f46e-afe9-11ec-b141-8ed1e89c66d6
    sourceVolumeMode: Filesystem
    volumeSnapshotClassName: csi-hostpath-snapclass-v1
...

4. Verify logs of snapshot-controller:

I0404 08:10:53.813378       1 snapshot_controller.go:698] snapcontent snapcontent-bbf7be38-ccb9-46bf-a420-f1a436bf10e4 has volume mode Filesystem

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Changes to snapshot controller to add SourceVolumeMode to VolumeSnapshotContents

[RaunakShah]

/hold until #683 is merged

[RaunakShah]

/hold until #683 is merged

[RaunakShah]

/hold cancel

[xing-yang]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RaunakShah, xing-yang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [xing-yang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[xing-yang]

Can you backport this to release-6.0?