Elasticsearch Query - Metric Results - Naming Convention - Count, Count1, Count2, etc,. Causing Issues

[mavasaf]

What happened:
Elasticsearch Query - Metric Results - Naming Convention - Count, Count1, Count2, etc,. Causing Issues especially when there is no data for one of the metric.
And in turn this is causing issue in the dashboard in displaying incorrect metric data and naming.

For Example.
Let's say, I have 3 Metrics Query from Elasticsearch Data. And I want to display them in the table panel.
Count - ( with a query A, Received Tickets)
Count - ( with a query B, Acknowledged Tickets)
Count - ( with a query C, Resolved Tickets)

Now in the transform window, I get the options as Count, Count1 And Count2.
And then I rename them appropriately to Business naming Conventions. ( in my case they are - PagerDuty Received Tickets, Acknowledged Tickets, Resolved Tickets)

Table Transformation 1 - ( use outer join with a common field to connect the queries A,B,C to give the result in one view)
Table Transformation 2 - ( Organize and rename )
Count - Received Tickets
Count1 - Acknowledged Tickets
Count2 - Resolved Tickets

Now when the panel refresh, when there is no data for one of the values, let's say received tickets( Query A, gives no response)
And hence we get only Count and Count1,. SInce we have the transform as above.
Count - Received Tickets
Count1 - Acknowledged Tickets.

So the panel shows data for Received and Acknowledged Tickets. whereas, the panel is supposed to show the acknowledged and resolved tickets only, since there is no received tickets.

What you expected to happen:

Here in this case, if we can have a naming convention that can be set by the user for each metric ?

How to reproduce it (as minimally and precisely as possible):
Please refer the above section for detailed steps.

Anything else we need to know?:
This above is only a sample case I have shared, we are experiencing this across many dashboard and panel. Any quick work around or solution is highly appreciated, thank you !

Environment: LOCAL, EPhemeral, NPE, PROD

• Grafana version: Latest ( 7.5.5)
• Data source type & version: Elasticsearch 6+ version
• OS Grafana is installed on: Linux and Windows.
• User OS & Browser: Windows, Chrome.
• Grafana plugins: N/A
• Others: N/A

[Elfo404]

Hi @mavasaf , does setting an Alias pattern per query work for you? It might be what you are looking for:

[mavasaf]

Thank you for the response in here,
I believe this holds good when there is Date Histogram.
However, when there is no Time line series, just group by Histogram, we are still seeing the issue.

[mavasaf]

Adding the details in here for visibility,.
What happened:
Elasticsearch Query - Metric Results - Naming Convention - Count, Count1, Count2, etc,. Causing Issues especially when there is no data for one of the metric.
And in turn this is causing issue in the dashboard in displaying incorrect metric data and naming.

For Example.
Let's say, I have 3 Metrics Query from Elasticsearch Data. And I want to display them in the table panel.
Count - ( with a query A, Received Tickets)
Count - ( with a query B, Acknowledged Tickets)
Count - ( with a query C, Resolved Tickets)

Now in the transform window, I get the options as Count, Count1 And Count2.
And then I rename them appropriately to Business naming Conventions. ( in my case they are - PagerDuty Received Tickets, Acknowledged Tickets, Resolved Tickets)

Table Transformation 1 - ( use outer join with a common field to connect the queries A,B,C to give the result in one view)
Table Transformation 2 - ( Organize and rename )
Count - Received Tickets
Count1 - Acknowledged Tickets
Count2 - Resolved Tickets

Now when the panel refresh, when there is no data for one of the values, let's say received tickets( Query A, gives no response)
And hence we get only Count and Count1,. SInce we have the transform as above.
Count - Received Tickets
Count1 - Acknowledged Tickets.

So the panel shows data for Received and Acknowledged Tickets. whereas, the panel is supposed to show the acknowledged and resolved tickets only, since there is no received tickets.

What you expected to happen:

Here in this case, if we can have a naming convention that can be set by the user for each metric ?

How to reproduce it (as minimally and precisely as possible):
Please refer the above section for detailed steps.

Anything else we need to know?:
This above is only a sample case I have shared, we are experiencing this across many dashboard and panel. Any quick work around or solution is highly appreciated, thank you !

Environment: LOCAL, EPhemeral, NPE, PROD

Grafana version: Latest ( 7.5.5)
Data source type & version: Elasticsearch 6+ version
OS Grafana is installed on: Linux and Windows.
User OS & Browser: Windows, Chrome.
Grafana plugins: N/A
Others: N/A

[gabor]

hmmm.. i think what we need is not even elasticsearch-specific.. some way to refer to the query-name (the thing that starts with A by default) in a transformation.. what do you think @grafana/user-essentials ? (at least i guess that's the right squad, if not, please forward me 😄 )

[JoaoSilvaGrafana]

I was able to reproduce this, your idea of keeping the query name could work, but unfortunately the "Outer Join" transformation loses all of that at the moment.
I am trying to understand why Gio's workaround of adding alias for each series doesn't work. @mavasaf , did you ever find a solution to this? And if that workaround still does not work could you share a dashboard json explaining why it doesn't work? Thanks!

[gabor]

@JoaoSilvaGrafana i think the problem with Gio's workaround is that the alias-field is available when you use the "date histogram" mode, but it is not available when you choose "histogram".

so, in theory a potential fix is to allow the "alias" also for "histogram" (i'm not sure if this can work or not, would require further investigation), it's just to me it seems best would be to solve these issues somewhere "higher", so that every datasource does not have to implement alias-functionality again and again.

[JoaoSilvaGrafana]

Ah ok I did not get that point that not all datasources have "alias". This will be slightly complicated to fix from the transformations side, because by the time the transformation gets the metrics, we no longer have the series "names" (A, B, C...), so would need to figure out at what stage to pass that information to the transformation.
Since we have split triage I will let someone from @grafana/dashboards-squad look into this and move it to their board to look at it, but I also think this could be something we do as mob? 😄

[gabor]

i'm willing to help in any way necessary if it helps with not having to implement the alias-functionality in every datasource separately 😄

[benmelnick]

Have there been any updates on this issue? My team owns a number of Grafana dashboards that we use for critical system insights, but because of this issue we sometimes cannot easily tell if our metrics are accurate or not. Fixing this problem would solve a huge monitoring issue for us.