-
Notifications
You must be signed in to change notification settings - Fork 473
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[gardenlet] Switch
BackupBucket controller to controller-runtime (#…
…6837) * Add documentation * Use clock.Clock in the reconciler * Skip queuing on update if `newBackupBucket.Generation == oldBackupBucket.Generation` This was done because the earlier comparison of `status.ObservedGeneration` caused multiple reconciliations due to the Status updation by the flow progressReporter. Later, after refactoring, `GenerationChangedPredicate` will be used. * Add integration tests * Use spec.seedName field selector for BackupBucket cache This way, we don't have to filter unrelated `BackupBucket`s at the controller level (e.g., with predicates). The underlying controller-runtime cache already makes sure that we only see objects which are related to our seed now. * Refactor backupbucket controller to native controller-runtime controller * Drop no longer needed code * Add spec.bucketName field selector to BackupEntry * Minor code changes * Fix failing tests * Rename files * Address review comments * [Address review comments] Do not hardcode Garden namespace name * Use ExpectWithOffSet and DeferCleanup * Remove acuator and move code to reconciler * Move seed creation to BeforeSuite * Address PR review feedback * Use clock.Clock in reconciler for BackupEntry controller * Add unit tests for eventhandler * Do no restrict gardenlet's cache * Watch extension BackupBuckets and map to BackupBuckets * Eliminate wait calls for extension * Improve integration tests * Address PR review feedback * Address PR review feedback: Improve predicates * Only reconcile extension backupbucket when state is failed * Reuse v1betahelper.HasOperationAnnotation function * Address PR review feedback * Move ExtensionStatusChanged predicate to predicate utils * Minor nits stress -ignore "unable to grab random port" -p 32 ./test/integration/gardenlet/backupbucket/backupbucket.test ─╯ Before: ``` 3m55s: 391 runs so far, 2 failures (0.51%) 4m0s: 403 runs so far, 2 failures (0.50%) 4m5s: 413 runs so far, 2 failures (0.48%) ``` After: ``` 7m35s: 728 runs so far, 0 failures 7m40s: 729 runs so far, 0 failures 7m45s: 740 runs so far, 0 failures 7m50s: 743 runs so far, 0 failures 7m55s: 760 runs so far, 0 failures 8m0s: 761 runs so far, 0 failures 8m5s: 770 runs so far, 0 failures ``` * Update status.ObservedGeneration at the beginning of reconciliation * Requeue extension on Create only if State is failed * Add dependencies in skaffold.yaml * Apply suggestions from code review * Remove operation annotation in the apiserver * Deflake integration tests ``` stress -ignore "unable to grab random port" -p 32 ./test/integration/gardenlet/backupbucket/backupbucket.test ... 26m10s: 1978 runs so far, 0 failures 26m15s: 1987 runs so far, 0 failures 26m20s: 1993 runs so far, 0 failures 26m25s: 1998 runs so far, 0 failures 26m30s: 2004 runs so far, 0 failures 26m35s: 2010 runs so far, 0 failures ```
- Loading branch information
Showing
36 changed files
with
1,848 additions
and
928 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| // Copyright (c) 2022 SAP SE or an SAP affiliate company. All rights reserved. This file is licensed under the Apache Software License, v. 2 except as noted otherwise in the LICENSE file | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package controllerutils | ||
|
|
||
| import ( | ||
| "time" | ||
|
|
||
| "k8s.io/apimachinery/pkg/types" | ||
| "k8s.io/client-go/util/workqueue" | ||
| "k8s.io/utils/clock" | ||
| "sigs.k8s.io/controller-runtime/pkg/client" | ||
| "sigs.k8s.io/controller-runtime/pkg/event" | ||
| "sigs.k8s.io/controller-runtime/pkg/handler" | ||
| "sigs.k8s.io/controller-runtime/pkg/reconcile" | ||
|
|
||
| gardencorev1beta1 "github.com/gardener/gardener/pkg/apis/core/v1beta1" | ||
| ) | ||
|
|
||
| // EnqueueCreateEventsOncePer24hDuration returns handler.Funcs which enqueues the object for Create events only once per 24h. | ||
| // All other events are normally enqueued. | ||
| func EnqueueCreateEventsOncePer24hDuration(clock clock.Clock) handler.Funcs { | ||
| return handler.Funcs{ | ||
| CreateFunc: func(evt event.CreateEvent, q workqueue.RateLimitingInterface) { | ||
| if evt.Object == nil { | ||
| return | ||
| } | ||
| q.AddAfter(reconcile.Request{NamespacedName: types.NamespacedName{ | ||
| Name: evt.Object.GetName(), | ||
| Namespace: evt.Object.GetNamespace(), | ||
| }}, getDuration(evt.Object, clock)) | ||
| }, | ||
| UpdateFunc: func(evt event.UpdateEvent, q workqueue.RateLimitingInterface) { | ||
| if evt.ObjectNew == nil { | ||
| return | ||
| } | ||
| q.Add(reconcile.Request{NamespacedName: types.NamespacedName{ | ||
| Name: evt.ObjectNew.GetName(), | ||
| Namespace: evt.ObjectNew.GetNamespace(), | ||
| }}) | ||
| }, | ||
| DeleteFunc: func(evt event.DeleteEvent, q workqueue.RateLimitingInterface) { | ||
| if evt.Object == nil { | ||
| return | ||
| } | ||
| q.Add(reconcile.Request{NamespacedName: types.NamespacedName{ | ||
| Name: evt.Object.GetName(), | ||
| Namespace: evt.Object.GetNamespace(), | ||
| }}) | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| func getDuration(obj client.Object, clock clock.Clock) time.Duration { | ||
| switch obj := obj.(type) { | ||
| case *gardencorev1beta1.BackupBucket: | ||
| return ReconcileOncePer24hDuration(clock, obj.ObjectMeta, obj.Status.ObservedGeneration, obj.Status.LastOperation) | ||
| case *gardencorev1beta1.BackupEntry: | ||
| return ReconcileOncePer24hDuration(clock, obj.ObjectMeta, obj.Status.ObservedGeneration, obj.Status.LastOperation) | ||
| } | ||
| return 0 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| // Copyright (c) 2022 SAP SE or an SAP affiliate company. All rights reserved. This file is licensed under the Apache Software License, v. 2 except as noted otherwise in the LICENSE file | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package controllerutils_test | ||
|
|
||
| import ( | ||
| "time" | ||
|
|
||
| gardencorev1beta1 "github.com/gardener/gardener/pkg/apis/core/v1beta1" | ||
| . "github.com/gardener/gardener/pkg/controllerutils" | ||
| "github.com/gardener/gardener/pkg/utils/test" | ||
|
|
||
| . "github.com/onsi/ginkgo/v2" | ||
| . "github.com/onsi/gomega" | ||
| metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
| "k8s.io/apimachinery/pkg/types" | ||
| "k8s.io/client-go/util/workqueue" | ||
| testclock "k8s.io/utils/clock/testing" | ||
| "sigs.k8s.io/controller-runtime/pkg/event" | ||
| "sigs.k8s.io/controller-runtime/pkg/handler" | ||
| "sigs.k8s.io/controller-runtime/pkg/reconcile" | ||
| ) | ||
|
|
||
| var _ = Describe("EventHandler", func() { | ||
| Describe("#EnqueueCreateEventsOncePer24hDuration", func() { | ||
| var ( | ||
| handlerFuncs = handler.Funcs{} | ||
| queue workqueue.RateLimitingInterface | ||
| fakeClock *testclock.FakeClock | ||
|
|
||
| backupBucket *gardencorev1beta1.BackupBucket | ||
| ) | ||
|
|
||
| BeforeEach(func() { | ||
| fakeClock = testclock.NewFakeClock(time.Now()) | ||
| queue = workqueue.NewRateLimitingQueueWithDelayingInterface(workqueue.NewDelayingQueueWithCustomClock(fakeClock, ""), workqueue.DefaultControllerRateLimiter()) | ||
| handlerFuncs = EnqueueCreateEventsOncePer24hDuration(fakeClock) | ||
|
|
||
| backupBucket = &gardencorev1beta1.BackupBucket{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Name: "bar", | ||
| Namespace: "foo", | ||
| Generation: 1, | ||
| }, | ||
| Status: gardencorev1beta1.BackupBucketStatus{ | ||
| ObservedGeneration: 1, | ||
| LastOperation: &gardencorev1beta1.LastOperation{ | ||
| Type: gardencorev1beta1.LastOperationTypeReconcile, | ||
| State: gardencorev1beta1.LastOperationStateSucceeded, | ||
| LastUpdateTime: metav1.NewTime(fakeClock.Now()), | ||
| }, | ||
| }, | ||
| } | ||
| }) | ||
|
|
||
| It("should enqueue a Request with the Name / Namespace of the object in the CreateEvent immediately if the last reconciliation was 24H ago", func() { | ||
| evt := event.CreateEvent{ | ||
| Object: backupBucket, | ||
| } | ||
| fakeClock.Step(24 * time.Hour) | ||
| handlerFuncs.Create(evt, queue) | ||
| verifyQueue(queue) | ||
| }) | ||
|
|
||
| It("should enqueue a Request with the Name / Namespace of the object in the CreateEvent after a random duration if the last reconciliation was not 24H ago", func() { | ||
| DeferCleanup(test.WithVars(&RandomDuration, func(time.Duration) time.Duration { return 250 * time.Millisecond })) | ||
| evt := event.CreateEvent{ | ||
| Object: backupBucket, | ||
| } | ||
| handlerFuncs.Create(evt, queue) | ||
| Expect(queue.Len()).To(Equal(0)) | ||
| fakeClock.Step(1 * time.Second) | ||
| Eventually(func() int { | ||
| return queue.Len() | ||
| }).Should(Equal(1)) | ||
| verifyQueue(queue) | ||
| }) | ||
|
|
||
| It("should enqueue a Request with the Name / Namespace of the object in the UpdateEvent.", func() { | ||
| evt := event.UpdateEvent{ | ||
| ObjectNew: backupBucket, | ||
| ObjectOld: backupBucket, | ||
| } | ||
| handlerFuncs.Update(evt, queue) | ||
| verifyQueue(queue) | ||
| }) | ||
|
|
||
| It("should enqueue a Request with the Name / Namespace of the object in the DeleteEvent.", func() { | ||
| evt := event.DeleteEvent{ | ||
| Object: backupBucket, | ||
| } | ||
| handlerFuncs.Delete(evt, queue) | ||
| verifyQueue(queue) | ||
| }) | ||
| }) | ||
| }) | ||
|
|
||
| func verifyQueue(queue workqueue.RateLimitingInterface) { | ||
| ExpectWithOffset(1, queue.Len()).To(Equal(1)) | ||
|
|
||
| i, _ := queue.Get() | ||
| ExpectWithOffset(1, i).NotTo(BeNil()) | ||
| req, ok := i.(reconcile.Request) | ||
| ExpectWithOffset(1, ok).To(BeTrue()) | ||
| ExpectWithOffset(1, req.NamespacedName).To(Equal(types.NamespacedName{ | ||
| Name: "bar", | ||
| Namespace: "foo", | ||
| })) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.