PLNSRVCE-1692: consolidate metrics into single pipelinerun, taskrun, pod controllers tuned for performance; convert pvc metric to poll only #62
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… tuned for performance; convert pvc metric to poll only; more clean up and reorg As part of getting ready to add the 'no pod create attempts occuring for pipelineruns' metric for satisfying AppSRE/Infra feedback on core tekton health, I've been monitoring the memory usage on prod-r01 and the previous "simpler" approach of creating controllers per replicas may be now proving untenable. As konflux onboading increases, exporter's memory usage continues to grow, and I've had to increase the memory limit from 1G to 3G just in the last few weeks. Note, the consolidating metrics into common reconcilers made the PVC gauge metric utilizing a reconciler in addition to the existing poller (a la what upstream does) untenable. Hence we now just leverage the polling thread for the PVC metric. rh-pre-commit.version: 2.2.0 rh-pre-commit.check-secrets: ENABLED
rh-pre-commit.version: 2.2.0 rh-pre-commit.check-secrets: ENABLED
gabemontero
changed the title
|
@jkhelil - Romain gave me admin perms and I've added you to the repo. You're now listed as an official reviewer on this PR |
jkhelil
approved these changes
Apr 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As part of getting ready to add the 'no pod create attempts occuring for pipelineruns' metric for satisfying AppSRE/Infra feedback on core tekton health, I've been monitoring the memory usage on prod-r01 and the previous "simpler" approach of creating controllers per replicas may be now proving untenable. As konflux onboading increases, exporter's memory usage continues to grow, and I've had to increase the memory limit from 1G to 3G just in the last few weeks.
Note, the consolidating metrics into common reconcilers made the PVC gauge metric utilizing a reconciler in addition to the existing poller (a la what upstream does) untenable. Hence we now just leverage the polling thread for the PVC metric.
rh-pre-commit.version: 2.2.0
rh-pre-commit.check-secrets: ENABLED
@enarha @jkhelil PTAL
@jkhelil your github ID does not show up as a reviewer. My guess is there is some github related onboarding activity you still have to do, though I'm not sure what it is exactly. See if you can find it in the onboarding doc, or maybe see if Romain knows.