Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: migrate controllers to SSA #112

Merged
merged 1 commit into from
Feb 16, 2024
Merged

Conversation

erikgb
Copy link
Contributor

@erikgb erikgb commented Dec 8, 2023

This PR migrates all updates in controllers to server-side apply (SSA), except the update removing the finalizer on SubNamespace - which is changed to a standard patch.

SSA works a bit differently than updates, so some of the optimizations had to go in this PR. I am hoping this change won't increase the load on the api-server significantly, but it's something we should monitor. I have added stable resource version tests for namespaces - which are now SSAed unconditionally.

Namespace controller tests are currently highly scenario-based, which makes them long and hard to read/modify. I have added some additional assertions of modified behavior on migration to SSA, but I suggest refactoring the tests in a follow-up PR to improve the test coverage. I can prepare a PR if you agree.

Fixes #98

@erikgb erikgb force-pushed the ssa-migration branch 2 times, most recently from 87f5663 to a8dbfaa Compare December 10, 2023 11:35
@erikgb erikgb changed the title WIP: chore: migrate controllers to SSA chore: migrate controllers to SSA Dec 10, 2023
@erikgb erikgb marked this pull request as ready for review December 10, 2023 11:35
@erikgb erikgb marked this pull request as draft December 14, 2023 13:11
@erikgb erikgb changed the title chore: migrate controllers to SSA WIP: chore: migrate controllers to SSA Dec 14, 2023
@erikgb
Copy link
Contributor Author

erikgb commented Dec 14, 2023

After testing this PR I think it needs a bit more work - especially to avoid endless reconcile loops.

@erikgb erikgb changed the title WIP: chore: migrate controllers to SSA chore: migrate controllers to SSA Feb 2, 2024
@erikgb erikgb marked this pull request as ready for review February 2, 2024 22:55
@erikgb
Copy link
Contributor Author

erikgb commented Feb 2, 2024

I think this PR is ready for review now. After we hopefully get this in, I am hoping to get a new release soon. 🙏 There are already many interesting unreleased improvements, and fixing #98 (this PR) is a high priority for us.

While you review this PR, I will prepare some vanilla release preparation PRs like dumping dependencies etc. Have you ever considered enabling Depandabot/Renovate on this project?

@yamatcha
Copy link
Contributor

yamatcha commented Feb 9, 2024

@erikgb

Namespace controller tests are currently highly scenario-based, which makes them long and hard to read/modify. I have added some additional assertions of modified behavior on migration to SSA, but I suggest refactoring the tests in a follow-up PR to improve the test coverage. I can prepare a PR if you agree.

Current test is the scenario we need in our environment, so we don't want to lose some of them in the major refactoring.
Would you tell me how you want to refactor this.

@@ -30,7 +30,7 @@ metadata:
team: foo
```

Accurate only propagates labels/annotations that have been configured in that respect via the `labelKeys` and `annotationKeys` parameters in `config.yaml`. This prevents the propagation of labels/annotations that were not meant to do so. Accurate currently does not delete previously propagated labels when deleted from the parent namespace to prevent unintended deletions. Users are expected to manually delete labels/annotations that are no longer needed.
Accurate only propagates labels/annotations that have been configured in that respect via the `labelKeys` and `annotationKeys` parameters in `config.yaml`. This prevents the propagation of labels/annotations that were not meant to do so.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR includes a change in specifications.
Could you please tell me why you are changing this behavior?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yamatcha
This is the very reason why @erikgb is making this PR.
Without SSA, it was difficult to remove labels and annotations safely.
Now with SSA, removing them is a safe operation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly! @ymmt2005 is correct that this is the main motivation for this PR. There is yet no 1. class support for SSA in controller-runtime, but I have been involved in migrating several operators to SSA using the same approach as in this PR. So I am pretty sure it works, and it's also properly tested now IMO. The main concern with SSA migration is the risk of getting never-ending reconcile loops, but I have added new tests for this.

We would be extremely happy if this PR could eventually get merged and released. #98 is a semi-blocker for our further rollout of Accurate in our clusters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erikgb
Thank you for the detailed description!
I missed issue refered in PR description.

@erikgb
Copy link
Contributor Author

erikgb commented Feb 9, 2024

@erikgb

Namespace controller tests are currently highly scenario-based, which makes them long and hard to read/modify. I have added some additional assertions of modified behavior on migration to SSA, but I suggest refactoring the tests in a follow-up PR to improve the test coverage. I can prepare a PR if you agree.

Current test is the scenario we need in our environment, so we don't want to lose some of them in the major refactoring. Would you tell me how you want to refactor this.

This was just a suggestion and not a requirement. I have no problems leaving the tests as-is. But if I were a maintainer, I would split things more. I don't see a big benefit in the long scenario-based tests, and think the test code would be easier to read if the scenarios were tested one by one. The extensive use of Ginkgo By construct is a sign of this (but required for these lengthy tests). 😄 No offense, of course. This is just my point of view.

@@ -462,4 +476,60 @@ var _ = Describe("Namespace controller", func() {
HaveField("Annotations", HaveKeyWithValue("memo", "tama")),
))
})

Context("templated namespace", func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you want this test to confirm?
I think confirming that ns is created from template and that labels and annotations are propagated has already been tested here.

Expect(ns1.Labels).To(HaveKeyWithValue("foo.bar/baz", "baz"))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is to ensure consistent reconciles. Reconcile loops is the one thing to fear when migrating to SSA.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks to you, I understood.

Copy link
Contributor

@yamatcha yamatcha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yamatcha yamatcha merged commit 36bb411 into cybozu-go:main Feb 16, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Should clean up previously propagated namespace labels/annotations
3 participants