Significantly reduce shared informer memory usage #4012
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dcbw
force-pushed
the
save-informer-memory
branch
from
93a9bac
to
231847d
Compare
dcbw
force-pushed
the
save-informer-memory
branch
2 times, most recently
from
e7101f1
to
d9a7732
Compare
dcbw
force-pushed
the
save-informer-memory
branch
from
d9a7732
to
e17e8f2
Compare
|
makes sense to me, but |
|
@npinaeva CI look better now? |
dcbw
force-pushed
the
save-informer-memory
branch
from
e17e8f2
to
0e6d212
Compare
kubernetes/kubernetes#118455 Kubernetes-commit: c86f562f29b6b7498ea962d2ac596e6d26ecd723 Kubernetes-commit: 9c4651bd0479b84f5e5913649207476717f3f13e Signed-off-by: Eric Lin <exlin@google.com>
We don't care about them, so why cache them and use a ton of memory. Inspired by kubernetes/kubernetes#118455 Signed-off-by: Dan Williams <dcbw@redhat.com>
…thRetry() function Signed-off-by: Dan Williams <dcbw@redhat.com>
dcbw
force-pushed
the
save-informer-memory
branch
from
0e6d212
to
51fc946
Compare
|
@npinaeva CI all green on first try now |
This was referenced Nov 28, 2023
|
Fix included in accepted release 4.15.0-0.nightly-2023-12-02-123536 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Kubernetes client-go v0.29 (not yet released) adds a WithTransform() option to shared informer factories that lets callers modify objects before they are added to the cache. Since we cache lots of objects (every pod, service, endpoint, node, etc) we can nil out a bunch of fields we don't care about, and save a lot of memory. This reduces ovn-kube memory usage by about 30% at high scale (250 nodes, 60,000 pods, 30,000 services)
The only catch is we need to carry a client-go patch until we can bump to v0.29 next year. But it's really small.