TopN should return no more than N resources

[cgrinds]

Volume dashboard is a good way to reproduce
Commit: 702d351

If you select top 5, Prometheus returns more than 5 resources to Grafana. Looks like a known issue with a workaround.

• https://grafana.com/docs/grafana/latest/datasources/prometheus/ has more details

[Hardikl]

Issue: In case of range vectors, topk function would give N records at each time-stamp basis, not at consolidated to whole given range, which result in having more than N records. For Instant vector case, topk would be working as expected and give exact N records.

topN investigation so far:

• We haven't found any grafana dashboard from catalog which has topk var, P95/P98 var is available, but it's direct query which gives the value
• Panel level variable is not supported in grafana
• We worked together and identified the query which gives top n volume names, but join is not possible in Prometheus
• Then, we tried to join range with instant vector in grafana, but [range (op) instant = range] turn into range, which is same as more than N records, and not solving the issue in grafana
• One option is to pass this identified top n volume names in range series - which is same as introducing new global variable for each query - which led us to have many hidden variables, same as PR fix: handle topk: volume POC #708
• There are few known issues and links which tells that VictoriaMetrics supports the same, not PromQL. And few link would still point to workaround which is in PR.
  https://stackoverflow.com/questions/66016655/can-you-apply-a-limit-on-promql-query
  /graph needs a way to select the "top 5" or similar prometheus/prometheus#586
  https://stackoverflow.com/questions/43840372/how-to-show-top-10-values-only-in-grafana

workaround: https://www.robustperception.io/graph-top-n-time-series-in-grafana

Thanks @rahulguptajss and @cgrinds for the support.

[cgrinds]

@Hardikl updated the wiki with workaround info, closing