Allow template variable substitution in panel promql

[arajkumar]

As mentioned in the $title, this PR enables templated variable substitution in panel promql expressions.

Example: https://github.com/kubernetes-monitoring/kubernetes-mixin/blob/master/dashboards/network-usage/cluster-total.libsonnet#L395

Signed-off-by: Arunprasad Rajkumar arajkuma@redhat.com

[arajkumar]

Unrelated info: We have recently added dashboard linting to k8s-mixin project.
kubernetes-monitoring/kubernetes-mixin#736

[tomwilkie]

Hi @arajkumar - thank you for your PR! We're working on this exact thing in #17 - does that fix your problem?

In this case, does $interval:$resolution actually get correctly substituted by Grafana? I don't see them in list of global variables: https://grafana.com/docs/grafana/latest/variables/variable-types/global-variables/

[tomwilkie]

@wuzuf would you mind figuring out if these variables should be supported too?

[arajkumar]

@tomwilkie @wuzuf This PR is actually about supporting Grafana custom variables used in promql expressions., not related to Grafana global variable. Even after #17 , promql expression rate(http_requests_total[$interval:$resolution]) is still failing to pass through the linter.($interval and $resolution are template variables)

Here is a workaround we have applied on k8s-mixin to mitigate this problem.

[wuzuf]

Indeed, I think the challenge here is to find the right "sample value" for a variable depending on where it is used inside the promQL expression. I see two options:

1. We try to be "context aware" and replace $interval with something that looks like a duration and $resolution with an int. That means rewriting a promQL parser more or less. Probably a bit complexe
2. We parse also the template variables defined in the dashboard and we pick the variable value from the dashboard itself (assuming those are not coming from a datasource). That would make us very close to having a check that variables used are all defined somewhere, too.

@arajkumar would you by any chance be in a position to share your dashboard json? I would not mind exploring options 2.

[arajkumar]

Hey @wuzuf, Thanks for the explanation.

Here is a sample dashboard from k8s-mixins which uses variable beyond metric label selection: https://github.com/kubernetes-monitoring/kubernetes-mixin/blob/master/dashboards/network-usage/cluster-total.libsonnet#L395

[tomwilkie]

@tomwilkie @wuzuf This PR is actually about supporting Grafana custom variables used in promql expressions., not related to Grafana global variable. Even after #17 , promql expression rate(http_requests_total[$interval:$resolution]) is still failing to pass through the linter.($interval and $resolution are template variables)

Ah I didn't clock that. Why does this dashboard need a template variable for the interval? Does it make more sense to use the $__rate_interval provided by Grafana?

[arajkumar]

@tomwilkie

Ah I didn't clock that. Why does this dashboard need a template variable for the interval? Does it make more sense to use the $__rate_interval provided by Grafana?

IIUC, $interval template variable shall be replaced with global variable $__rate_interval. But still subquery template variable $resolution will get into same problem.

[arajkumar]

Closing in favour of #19