Warning "CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+" when installing rancher backups

[sowmyav27]

Describe the bug
Warning "CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+" when installing rancher backups

To Reproduce
Installed on an RKE1 cluster using rke: 1.3.0-rc14. Will happen on a rancher install 2.6-head 723f747 also

• Install rancher backup chart 2.0.0+up2.0.0-rc on an RKE1 cluster.
• Warning seen in the logs:

Result
Logs:

helm install rancher-backup-crd . -n cattle-resources-system --create-namespace
WARNING: "kubernetes-charts.storage.googleapis.com" is deprecated for "stable" and will be deleted Nov. 13, 2020.
WARNING: You should switch to "https://charts.helm.sh/stable" via:
WARNING: helm repo add "stable" "https://charts.helm.sh/stable" --force-update
W0810 21:21:24.832270   73017 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0810 21:21:24.915551   73017 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0810 21:21:24.999162   73017 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0810 21:21:25.282452   73017 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0810 21:21:25.294405   73017 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0810 21:21:25.355505   73017 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
NAME: rancher-backup-crd
LAST DEPLOYED: Tue Aug 10 21:21:24 2021
NAMESPACE: cattle-resources-system
STATUS: deployed
REVISION: 1

[cbron]

Moving this to 2.6.1 since we won't support 1.22 until later

[jiaqiluo]

we need to add the following to the release note and rancher docs:

Because k8s 1.22 drops the apiVersion apiextensions.k8s.io/v1beta1, trying to restore an existing backup file into a 1.22 cluster will fail because the backup file contains CRDs with the apiVersion v1beta1.

But the good news is that we update the default resourceSet to collect the CRDs with the apiVersion v1. So they can update their rancher-backup app, then make backups, new backups should work in k8s 1.22

[jiaqiluo]

Can be tested with the chart rancher-backup 2.1.0-rc1

Root cause

The initial issue is that CRDs in the rancher-backup-crd chart use the apiVersion apiextensions.k8s.io/v1beta1 which is removed from the k8s 1.22 cluster. apiextensions.k8s.io/v1 is the replacement.
Later, it turns out the default resoureSet still collects some CRDs with the apiVersionapiextensions.k8s.io/v1beta1, and the rancher-backup app still uses old modules with v1beta1 APIs, so the rancher-backup app will not be able to make or restore backups in the k8s 1.22 cluster properly.

What was fixed, or what changes have occurred

There are two fixes:

• update the CRDs in the rancher-backup-crd chart to the v1 apiVersion.
• update the default resouceSet and the client to use new APIs internally.

Areas or cases that should be tested

• fresh install of the app
• upgrade of the app
• make backups in 1.21/1.22 clusters
• restore backups in 1.21/1.22 clusters

Note:

• When making or restoring backups, rancher version and local cluster's k8s version should be the same.
• old backups made in 1.21 cluster will not work in 1.22 cluster because 1.22 drops the apiVersion apiextensions.k8s.io/v1beta1

What areas could experience regressions?

• installation and upgrading
• making and restoring backups

Are the repro steps accurate/minimal?

n/a

[jiaqiluo]

https://github.com/rancher/docs/issues/3694

[anupama2501]

Verified on v2.6.3-rc3 , backup version 2.1.0+up2.1.0-rc1

Fresh install of the app on local k8s v1.21.6-rancher1-2 with migration

1. Create a rancher server HA with local on v1.21.6-rancher1-2
2. Created 2 downstream rke1 node driver AWS and DO clusters with k8s v1.21 and v1.22
3. Deployed a few workloads and ingresses
4. Installed backup operator version 2.1.0-rc1
5. Verified the warning Warning "CustomResourceDefinition is deprecated in v1.16 is not seen.
6. Took backup b1 and stored in S3
7. Created an rke1 cluster on same k8s version and backup installed on it. - C1
8. Restored the clusters on to the new RKE1 c1 cluster via migration
9. Created a few workloads and ingresses and no errors were seen.

Fresh install of the app on local k8s v1.22.3-rancher1-1 with migration

Repeated the same steps as above on local cluster installed on k8s 1.22.3-rancher1-1

Upgrade of the app

1. Create a rancher server HA with local on v1.21.6-rancher1-2
2. Created 2 downstream rke1 node driver AWS and DO clusters with k8s v1.21 and v1.22
3. Deployed a few workloads and ingresses
4. Installed backup operator version 2.0.1
5. Warning Warning "CustomResourceDefinition is deprecated in v1.16 is seen.
6. Took backup b1 and stored in S3
7. Upgraded the backup version to 2.1.0-rc1
8. Restored the clusters on the same rancher server
9. Created a few workloads and ingresses and verified no errors were seen.

Restore in the same cluster

1. Create a rancher server HA with local on v1.21.6-rancher1-2
2. Created 2 downstream rke1 node driver AWS and DO clusters with k8s v1.21 and v1.22
3. Deployed a few workloads and ingresses
4. Installed backup operator version 2.1.0-rc1
5. Verified the warning Warning "CustomResourceDefinition is deprecated in v1.16 is not seen.
6. Took backup b1 and stored in S3
7. Restored the clusters on to the new RKE1 c1 cluster on the same local rke1 cluster.
8. Created a few workloads and ingresses and no errors were seen.