Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kube-prometheus-stack] kube-apiserver.rules is duplicate #1799

Closed
apricote opened this issue Feb 11, 2022 · 4 comments · Fixed by #2076
Closed

[kube-prometheus-stack] kube-apiserver.rules is duplicate #1799

apricote opened this issue Feb 11, 2022 · 4 comments · Fixed by #2076
Labels
bug Something isn't working help wanted Extra attention is needed lifecycle/stale

Comments

@apricote
Copy link

apricote commented Feb 11, 2022

Describe the bug a clear and concise description of what the bug is.

The kube-apiserver.rules rule group is duplicated by the kube-apiserver-burnrate.rules and kube-apiserver-histogram.rules rule groups.

In the upstream kubernetes-mixin, the kube-apiserver.rules were split into the other two in this commit: kubernetes-monitoring/kubernetes-mixin@2986c52

It looks like the two new files were added, but the old/deprecated file was not removed in #1165

This causes unnecessary CPU load in our kube-prometheus-stack installation. In total the rule evaluations take ~40seconds for each interval, of that kube-apiserver.rules takes 20seconds and -burnrate.rules + histogram.rules together take another 20seconds. By removing the old kube-apiserver.rules file, we could reduce or rule evalutions time by 50%.

What's your helm version?

v3.8.0

What's your kubectl version?

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"archive", BuildDate:"2022-01-27T18:26:18Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:19:12Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

Which chart?

kube-prometheus-stack

What's the chart version?

32.2.0

What happened?

Duplicate rules for apiserver_request:burnrateXX and cluster_quantile:apiserver_request_duration_seconds:histogram_quantile are deployed in the default configuration.

What you expected to happen?

I expected no duplicate rules to be deployed.

How to reproduce it?

Deploy the default chart and observe the duplicated rules in prometheus.

Enter the changed values of values.yaml?

NONE

Enter the command that you execute and failing/misfunctioning.

helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack

Command executes successfully, issue is with the deployed AlertRules.

Anything else we need to know?

No response

@stale
Copy link

stale bot commented Mar 13, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

@apricote
Copy link
Author

Issue is still there and relevant.

@stale stale bot removed the lifecycle/stale label Mar 17, 2022
@mrueg mrueg added the help wanted Extra attention is needed label Mar 17, 2022
@stale
Copy link

stale bot commented Apr 17, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

@stale
Copy link

stale bot commented May 1, 2022

This issue is being automatically closed due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed lifecycle/stale
Projects
None yet
2 participants