-
Notifications
You must be signed in to change notification settings - Fork 40.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve cadvisor (integration) tests #55398
Comments
/sig node |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
/remove-lifecycle stale |
@brancz Is that something you can get someone to look into? |
Unlikely, personally cAdvisor has been so unreliable for Prometheus metrics that we try not to rely on it and are even looking into getting the stats differently. Aside from that I would say this mostly belongs in the cAdvisor repo. |
@brancz No need to stablized cadvisor or improve that project itself, I was more thinking about better testing when bumping cadvisor version. And it's not cadvisor specific but something that applies to other metric related changes too (e.g #52121 and #53485). It just would be great to get people to pay more attention on metric changes in PRs. |
Understood. I don't have a great answer for you, yes I would love to, but don't have the resources to do this. Maybe it's a topic you can bring up to the CNCF CI working group? They already test Prometheus on Kubernetes, so maybe it's something they could look into (we're also working with them on some unrelated Prometheus CI things, maybe I can bring it up in there). |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
/remove-lifecycle stale |
So what are specific metrics that should be collected for now to close this issue? I guess the rest of metrics could be picked up when more issue arise. |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Not going to fight that bot anymore. |
Every other kubernetes release the metrics provided by cadvisor seem to break. This not only just happened again in #55397 but also earlier in #48483, #33192 and #25131.
These issue usually not only end up in the stable releases but even on cloud providers like GKE where these are sometimes the only supported versions.
I think this should be prevented by improving the cadvisor integration tests (if there are any yet). They should at least check that cadvisor successfully exposes plausible metrics for mem, cpu, disk io, filesystem and networking.
The text was updated successfully, but these errors were encountered: