ceph: update PodDisruptionBudget from v1beta1 to v1

[parth-gr]

This commit update the PodDisruptionBudget policy to use version v1
Updated to policy/v1 as policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+

Closes: #7917
Signed-off-by: parth-gr paarora@redhat.com

Description of your changes:

Which issue is resolved by this Pull Request:
Resolves #

Checklist:

• Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
• Skip Tests for Docs: Add the flag for skipping the build if this is only a documentation change. See here for the flag.
• Skip Unrelated Tests: Add a flag to run tests for a specific storage provider. See test options.
• Reviewed the developer guide on Submitting a Pull Request
• Documentation has been updated, if necessary.
• Unit tests have been added, if necessary.
• Integration tests have been added, if necessary.
• Pending release notes updated with breaking and/or notable changes, if necessary.
• Upgrade from previous release is tested and upgrade user guide is updated, if necessary.
• Code generation (make codegen) has been run to update object specifications, if necessary.

[parth-gr]

As discuss in this thread we need to wait for controller runtime 0.9 version for using v1.21 dependencies.

[travisn]

For the PDB version we likely need a similar approach to the CronJob, see this comment

[leseb]

FYI https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.9.0

[subhamkrai]

#8087

[subhamkrai]

We need to handle v1beta1 too.

[subhamkrai]

unit test failing

[travisn]

CI looks green, almost there!

[travisn]

@sp98 When did we get rid of the legacy PDBs? I think we can get rid of this method now since it's already been at least a release since we removed the legacy PDBs.

[sp98]

We added this in rook 1.5 in this PR. So should be safe to remove now.

[subhamkrai]

@sp98 We want to remove it in this PR or maybe in a separate PR?

[sp98]

separate PR, IMO.

[sp98]

nits

[sp98]

also, you need to see the upgrade scenario, see this comment from @sp98 on cronJob pr

@subhamkrai I have tested this situation by upgrading the k8s version(case 1.20.0 to 1.21.0).

The operator logs don't show any such Warnings.
But while calling the pdb pods it shows this :

@parth-gr - Were the pdbs moved from v1beta1 to v1 after you upgraded to 1.21?

**kubectl get pdb -nrook-ceph**
Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
NAME                MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
rook-ceph-mon-pdb   N/A             1                 1                     14m
rook-ceph-osd       N/A             1                 1                     18m

But after few minutes it goes away, I assume it is there till the time the cluster reconciles completely.
And rook-ceph-osd remains as it is.

EDIT : I looked into the operator log I see this warning too, as expected as we can not connect through the client while the cluster is down.

W0709 14:14:20.091591       9 reflector.go:436] pkg/mod/k8s.io/client-go@v0.21.2/tools/cache/reflector.go:167: watch of *v1.PodDisruptionBudget ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding
W0709 14:14:20.092998       9 reflector.go:436] pkg/mod/k8s.io/client-go@v0.21.2/tools/cache/reflector.go:167: watch of *v1beta1.PodDisruptionBudget ended with: an error on the server ("unable to decode an event from the watch stream: http2: client connection lost") has prevented the request from succeeding 

And these also goes away after the reconcile is completed.

[parth-gr]

@parth-gr - Were the pdbs moved from v1beta1 to v1 after you upgraded to 1.21?

Yes @sp98, pdbs are getting upgraded to v1.

 "apiVersion": "policy/v1",
  "kind": "PodDisruptionBudget",

But it takes one reconcile time to get the cluster in the ready state as we are upgrading, so the operator restarts and in between it's shows warning.

[leseb]

nit