clustermesh-apiserver: rework identities, endpoints and nodes synchronization to improve performance #25049
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
giorio94
added
area/clustermesh
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
7c3108f
to
82347ca
Compare
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
82347ca
to
91e3a44
Compare
giorio94
changed the title
giorio94
changed the title
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
91e3a44
to
f8fef13
Compare
|
/ci-multicluster |
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
5 times, most recently
from
b2edfa2
to
a0428a0
Compare
|
Force pushed to rebase onto main and resolve conflicts. |
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
3 times, most recently
from
e121ac1
to
7dd2cd5
Compare
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
7611b3c
to
466af29
Compare
maintainer-s-little-helper
bot
removed
the
dont-merge/needs-sign-off
|
Rebased onto main to fix conflicts. I've also added one additional commit to drop the temporary backend global variable introduced in #25554, since no longer necessary. |
|
/test |
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
466af29
to
8bf6550
Compare
|
/test |
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
8bf6550
to
e323efc
Compare
|
Rebased onto main to fix conflicts. |
|
/test Job 'Cilium-PR-K8s-1.26-kernel-net-next' failed: Click to show.Test NameFailure OutputJenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.26-kernel-net-next/384/ If it is a flake and a GitHub issue doesn't already exist to track it, comment Then please upload the Jenkins artifacts to that issue. |
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
e323efc
to
d3c19cb
Compare
|
Rebased onto main to fix conflicts. |
|
/test |
Let's change the NewWorkqueueSyncStore function to always take the clusterName as parameter, rather than relying on an option. This prevents inadvertently forgetting to set it. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
This commit extends the IPIdentityPair struct implementing the store.Key interface, to allow reading/writing it through the store abstractions. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
Add the CiliumSlimEndpointResource constructor, to be leveraged by the clustermesh-apiserver to improve the synchronization logic. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
This preparatory commit introduces the `synchronize` helper function to the clustermesh-apiserver, which will be subsequently used to uniform the synchronization process of the different resources from kubernetes to the kvstore. Essentially, the goal is to replace the vanilla informers with the "Resource[T]" abstraction, which better fits in the hive/cell paradigm and reduces the amount of boilerplate code. Each received event is then fed into the specific `synchronizer` implementation (to be introduced in the subsequent commits). It will take care of converting the object into the appropriate kvstore representation, and synchronizing it as required. Note: the synchronizer interface is not generic, since that approach turned out to cause issues with the `unused` linter, which failed to recognize the different type-specific implementations as actually used. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
This commit modifies the clustermesh-apiserver to read CiliumNodes through the Resource[T] abstraction rather than a vanilla informer, and synchronize them through the WorkqueueSyncStore. This ensures consistent behavior across all resource types (i.e., the retry logic is implemented by the SyncStore, rather than through the Resource[T] underlying workqueue). This approach has several advantages compared to the previous implementation, including: * Enables retrying kvstore operations in case of temporary failures; * Performs coalescencing in case of multiple updates for the same key; * Simplifies the overall logic, since conversions and sanity checks are already performed by the Resource[T] abstraction. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
This commit modifies the clustermesh-apiserver to read CiliumIdentities through the Resource[T] abstraction rather than a vanilla informer, and synchronize them through the WorkqueueSyncStore. This ensures consistent behavior across all resource types (i.e., the retry logic is implemented by the SyncStore, rather than through the Resource[T] underlying workqueue). This approach has several advantages compared to the previous implementation, including: * Enables retrying kvstore operations in case of temporary failures; * Performs coalescencing in case of multiple updates for the same key; * Simplifies the overall logic, since conversions and sanity checks are already performed by the Resource[T] abstraction. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
This commit modifies the clustermesh-apiserver to read CiliumEndpoints through the Resource[T] abstraction rather than a vanilla informer, and synchronize them through the WorkqueueSyncStore. This ensures consistent behavior across all resource types (i.e., the retry logic is implemented by the SyncStore, rather than through the Resource[T] underlying workqueue). This approach has several advantages compared to the previous implementation, including: * Enables retrying kvstore operations in case of temporary failures; * Performs coalescencing in case of multiple updates for the same key; * Simplifies the overall logic, since conversions and sanity checks are already performed by the Resource[T] abstraction. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
d3c19cb
to
b188c84
Compare
giorio94
removed
the
dont-merge/blocked
|
/test |
The temporary global backend variable is now only used in a very limited number of places. Let's propagate that parameter and drop the global variable. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
giorio94
force-pushed
the
mio/clustermesh-apiserver-resources
branch
from
b188c84
to
a4f3f26
Compare
|
/test |
|
All reviews from codeowners are in, and all tests are green. Marking as ready to merge |
giorio94
added
the
ready-to-merge
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR modifies the clustermesh-apiserver implementation to use the Resource[T] abstraction to retrieve CiliumIdentities, CiliumNodes and CiliumEndpoints from Kubernetes, rather than a vanilla informer, and synchronize them through the WorkqueueSyncStore. This ensures consistent behavior across all resource types (i.e., the retry logic is implemented by the SyncStore, rather than through the Resource[T] underlying workqueue).
This approach has several advantages, including:
Please, refer to the commit descriptions for more details about the individual changes.
Related: #25562