Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Resource[T] to implement CEP and CES watchers #29249

Conversation

pippolo84
Copy link
Member

@pippolo84 pippolo84 commented Nov 17, 2023

Use Resource[T] to implement the CiliumEndpoint and CiliumEndpointSlice k8s watchers. This removes the additional informers previously used, reducing the queries to the kube-apiserver and reducing the memory needed for the additional local caches.

Please refer to the individual commits for further details.

Depends on #29414, #29243, #29244 and #29246

Related: #28159

@pippolo84 pippolo84 added sig/k8s Impacts the kubernetes API, or kubernetes -> cilium internals translation layers. kind/performance There is a performance impact of this. release-note/misc This PR makes changes that have no direct user impact. area/modularization release-blocker/1.15 This issue will prevent the release of the next version of Cilium. labels Nov 17, 2023
@pippolo84

This comment was marked as outdated.

@pippolo84 pippolo84 force-pushed the pr/pippolo84/migrate-cilium-endpoint-resource branch from 8384ccc to 9fd0387 Compare November 18, 2023 10:23
@pippolo84 pippolo84 force-pushed the pr/pippolo84/migrate-cilium-endpoint-resource branch 2 times, most recently from d7ed88a to 715cb9e Compare November 28, 2023 16:50
@pippolo84 pippolo84 removed the release-blocker/1.15 This issue will prevent the release of the next version of Cilium. label Nov 28, 2023
@pippolo84

This comment was marked as outdated.

@pippolo84 pippolo84 added the dont-merge/blocked Another PR must be merged before this one. label Nov 29, 2023
@pippolo84 pippolo84 force-pushed the pr/pippolo84/migrate-cilium-endpoint-resource branch from 715cb9e to 8ea327b Compare November 29, 2023 11:23
@aanm aanm added dont-merge/wait-until-release Freeze window for current release is blocking non-bugfix PRs and removed dont-merge/blocked Another PR must be merged before this one. labels Dec 4, 2023
@pippolo84 pippolo84 added release-blocker/1.15 This issue will prevent the release of the next version of Cilium. and removed dont-merge/wait-until-release Freeze window for current release is blocking non-bugfix PRs labels Dec 13, 2023
@pippolo84 pippolo84 force-pushed the pr/pippolo84/migrate-cilium-endpoint-resource branch from 8ea327b to fddc505 Compare December 13, 2023 09:54
@pippolo84 pippolo84 force-pushed the pr/pippolo84/migrate-cilium-endpoint-resource branch from fddc505 to 0405ef0 Compare December 20, 2023 16:47
@pippolo84 pippolo84 marked this pull request as ready for review December 20, 2023 17:11
@pippolo84 pippolo84 requested review from a team as code owners December 20, 2023 17:11
pkg/k8s/watchers/cilium_endpoint.go Outdated Show resolved Hide resolved
pkg/k8s/watchers/cilium_endpoint.go Show resolved Hide resolved
@pippolo84 pippolo84 force-pushed the pr/pippolo84/migrate-cilium-endpoint-resource branch from 0405ef0 to 56b468e Compare December 22, 2023 16:53
Copy link
Member

@nathanjsweet nathanjsweet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@pippolo84
Copy link
Member Author

/test

The CEP and CES stores getter was originally meant for Hubble, but it is
not used anymore, so it is possible to clean it up.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Use a releasable Resource[CiliumSlimEndpoint] to refactor the agent CEP
watcher. Like the previous implementation, the underlying informer is
stopped as soon as the subscription to the events stream is canceled.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Use a releasable Resource[CiliumEndpointSlice] to refactor the agent CES
watcher. Like the previous implementation, the underlying informer is
stopped as soon as the subscription to the events stream is canceled.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
CiliumEndpoint and CiliumEndpointSlice resources are backed by a
releasable resource. To allow stopping the underlying informer when it
is not needed anymore, the cleanup routine should release the reference
to the store after collecting the stale CEPs.

Also, this keeps the CiliumEndpoint cleanup routine fully working even
when the agent is configured to use an external key-value store. In that
scenario, the routine starts the k8s informer to list the CEPs or CESs
and, once all the stale CEPs have been collected, releases the resource
to stop the informer.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
@pippolo84 pippolo84 force-pushed the pr/pippolo84/migrate-cilium-endpoint-resource branch from 56b468e to 62b287c Compare January 5, 2024 17:34
@pippolo84
Copy link
Member Author

/test

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Jan 5, 2024
@julianwiedmann julianwiedmann added this pull request to the merge queue Jan 9, 2024
@julianwiedmann
Copy link
Member

👋 if this is still a 1.15 release blocker, it will also need a 1.15 backport. Adding the label, please shout if it's no longer needded.

@julianwiedmann julianwiedmann added the needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch label Jan 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from main in v1.15.0-rc.1 Jan 9, 2024
Merged via the queue into cilium:main with commit 9c336fb Jan 9, 2024
62 checks passed
@jibi jibi mentioned this pull request Jan 12, 2024
32 tasks
@jibi jibi added backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. and removed needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch labels Jan 12, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.15 in v1.15.0-rc.1 Jan 12, 2024
@giorio94 giorio94 added backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. and removed backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. labels Jan 29, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.15 to Backport done to v1.15 in v1.15.0-rc.1 Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/modularization backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. kind/performance There is a performance impact of this. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-blocker/1.15 This issue will prevent the release of the next version of Cilium. release-note/misc This PR makes changes that have no direct user impact. sig/k8s Impacts the kubernetes API, or kubernetes -> cilium internals translation layers.
Projects
No open projects
v1.15.0-rc.1
Backport done to v1.15
Development

Successfully merging this pull request may close these issues.

None yet

7 participants