feat: Upgrade Azure Disk CSI driver to support various new features

[chewong]

Reason for Change:

• Upgraded the following CSI sidecar containers:
  • csi-provisioner v1.0.1 -> v1.4.0
  • csi-attacher v1.0.1 -> v1.2.0
  • azuredisk-csi-driver v0.3.0 -> v0.5.0
• Added the following CSI sidecar containers:
  • csi-snapshotter v1.1.0 (>= k8s 1.13)
  • csi-resizer v0.3.0 (>= k8s 1.16)
• Upgrading CSI containers enables the following new features:
  • Topology (k8s >= 1.14)
    • Requires volumeBindingMode: WaitForFirstConsumer and allowedTopologies key-value in storage class
    • Requires --feature-gates=Topology=true in csi-provisioner
  • Volume expansion (k8s >= 1.16)
    • Requires allowVolumeExpansion: true in storage class
  • Volume cloning (k8s >= 1.16)
  • Snapshot (k8s >= 1.13)
    • Requires --feature-gates=VolumeSnapshotDataSource=true in API server until >= k8s 1.17

Issue Fixed:

Requirements:

• uses conventional commit messages
• includes documentation
• adds unit tests
• tested upgrade from previous version

Notes:

[codecov]

Codecov Report

Merging #2541 into master will decrease coverage by 0.02%.
The diff coverage is 98.21%.

@@            Coverage Diff             @@
##           master    #2541      +/-   ##
==========================================
- Coverage   72.15%   72.12%   -0.03%     
==========================================
  Files         130      130              
  Lines       24486    24564      +78     
==========================================
+ Hits        17668    17717      +49     
- Misses       5797     5825      +28     
- Partials     1021     1022       +1

[chewong]

/cc @andyzhangx

[andyzhangx]

I think we could remove 1.13 support, to reduce support burden since we use different deployment scripts between 1.13 and 1.14+, other part LGTM

[chewong]

I think we could remove 1.13 support, to reduce support burden since we use different deployment scripts between 1.13 and 1.14+, other part LGTM

Since we still support creating 1.13.11 clusters, if we remove Azure Disk CSI driver, file.csi.azure.com is the only storage class available to use when cloud-controller-manager is enabled. Does that sound good?

[chewong]

@andyzhangx Should I also include kubernetes-sigs/azuredisk-csi-driver#252 into this PR?

[andyzhangx]

@andyzhangx Should I also include kubernetes-sigs/azuredisk-csi-driver#252 into this PR?

no, that PR is for next release. LGTM for this PR.

[andyzhangx]

/lgtm
/approve

[chewong]

/hold
Adding e2e tests.

[chewong]

Added cloud-controller-manager-enabled clusters as part of the e2e tests and ran on Jenkins against this PR - https://upstream.jenkins.azure-containers.io/view/aks-engine%20PRs/job/k8s-create-cluster-matrix/51/
@jackfrancis @mboersma @andyzhangx Could you take another look? Thanks.

[andyzhangx]

/lgtm
/approve

[jackfrancis]

/lgtm

[acs-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx, chewong, jackfrancis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jackfrancis]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment