e2e: deletion of CRDs may be delayed

[tmshort]

The e2e tests used to fail due to CRDs not being deleted in a timely manner. The CRDs would be in a terminating state with a specific error message. Eventually, after a few minutes, the CRDs are deleted.

This is handled in our e2e's by a 5-minute timeout while checking for CRD deletion.

This was the interesting condition on the CRDs:

{Terminating True 2024-11-06 21:00:31 +0000 UTC InstanceDeletionFailed could not list instances: storage is (re)initializing}

References to this error:

• https://github.com/kubernetes/enhancements/pull/4557/files
• https://github.com/kubernetes/kubernetes/pull/124642/files
• Error starting operator with K8s v1.31.0 (429 Message: storage is (re)initializing) java-operator-sdk#2526
• https://github.com/kubernetes/kubernetes/blob/e2bf630940946df5bc161d224e4a9b2e191a3b2e/staging/src/k8s.io/apiextensions-apiserver/pkg/controller/finalizer/crd_finalizer.go#L199-L204

Originally posted by @tmshort in #1428 (comment)

[github-actions]

Issues go stale after 90 days of inactivity. If there is no further activity, the issue will be closed in another 30 days.

[github-actions]

This issue has been closed due to inactivity.