Skip to content
This repository has been archived by the owner on Feb 22, 2022. It is now read-only.

[stable/prometheus-operator] Kubernetes Grafana dashboards missing interval variable causing some panels to be empty #22027

Closed
rayotte opened this issue Apr 20, 2020 · 5 comments
Labels
lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale.

Comments

@rayotte
Copy link

rayotte commented Apr 20, 2020

Describe the bug
When deploying prometheus operator, the Kubernetes dashboards that are included are now missing the "interval" template variable, which is causing the "current network usage" panel in the Kubernetes compute resource dashboards to appear empty as well as the "CPU Utilisation" panel in the k8's / compute Resources / cluster dashboard

Version of Helm and Kubernetes:
Helm client version: 2.15.1
Helm server version: 2.16.1
Kubernetes version: 1.16.2

Which chart:
Appears to be every [stable/prometheus-operator] chart after 8.12.4

What happened:
The panels described above used to show information when the interval variable was present and the promql expressions called it using [$interval]. Now the variable is not there and the expressions use [$__interval], casing the "Current network usage" panels to be blank

What you expected to happen:
These panels should be showing information like they were in previous releases

How to reproduce it (as minimally and precisely as possible):
Deploy the prometheus operator 8.13.0 chart, go to the grafana page and look at the "Kubernetes / Compute Resources / * " dashboards and look at "Current network usage" panels

@iangkent
Copy link

There seems to be constant regressions across prometheus-operator releases. Is there any testing done before the chart is released?

@den-is
Copy link
Contributor

den-is commented Apr 21, 2020

Compared to some other releases of this chart I've done test on completely fresh k8s install.
We just do sync with upstream https://github.com/coreos/kube-prometheus/tree/master/manifests or https://github.com/kubernetes-monitoring/kubernetes-mixin
They constantly introduce and deprecate resources.
So I completely feel what you are saying.
But you can fill a complaint there.
In the meantime, current prometheus-operator chart is working just fine for me.
And here is screenshot from that same fresh k8s setup with same fresh chart deploy:
Screen Shot 2020-04-21 at 10 11 37
k8s 1.18.0, helm 2.16.6

@den-is
Copy link
Contributor

den-is commented Apr 21, 2020

I another fresh reinstall but with helm3, just for fun.
And kinda encountered the same issue. :)
So I've decided to do some on surface research.
I'm using k8s 1.18, grafana 6.7.2, prom 2.17.2, chart 8.13.
Regarding $__interval variable - you can read about it here
googling "grafana $__interval not working" brought me to that interesting "issue" grafana/grafana#17924
Here are topics related to that change in the upstream: kubernetes-monitoring/kubernetes-mixin#372

Increasing range to 7days has shown values. Older setup will show data for smaller "range" (coz I never had "7 days" range, when was taking the previous screenshot)

@stale
Copy link

stale bot commented May 22, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

@stale stale bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 22, 2020
@stale
Copy link

stale bot commented Jun 5, 2020

This issue is being automatically closed due to inactivity.

@stale stale bot closed this as completed Jun 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale.
Projects
None yet
Development

No branches or pull requests

3 participants