Dashboard: Add support for Tempo query variables

[fabrizio-grafana]

What is this feature?
Add support for template variables (Label names and Label values) in Tempo, similarly to what is already available in Loki and Prometheus. Quick demo.

Why do we need this feature?
It has been requested by a few customers and discussed with @09jvilla.

Who is this feature for?
Customers.

Which issue(s) does this PR fix?:
Fixes #72575
Fixes #70305

[09jvilla]

Didn't review the code but approving based on the functionality demonstrated. Amazing work!

We have discussed in a separate thread that we should create another PR that ensures Tempo query variables work when the dashboard panel query is made via the Search tab. Today they only work if the query is made via the TraceQL tab.

[joey-grafana]

Could we modify the fields here so that there is one per row? We could also shorten the label width here as they seem quite long. In contrast, I think we should make the second fields dropdown input a bit longer. As you can see in the image, it's quite short which makes it hard to see what was selected in the dropdown.

[aocenas]

@fabrizio-grafana did you talk with somebody regarding whether we need or don't need also a search result (so TraceIDs) as a template variable?

[09jvilla]

@fabrizio-grafana did you talk with somebody regarding whether we need or don't need also a search result (so TraceIDs) as a template variable?

@aocenas -- This is something we're leaving out of scope for now and will revisit in the future if there is sufficient demand.

[joey-grafana]

This looks really good and will be a great addition :)

I've noticed something that we should fix before merging. Nothing major so should be ok to fix. Querying the v2 endpoint always returns an error because we are not providing the scope in the query.

i.e. the query is like api/v2/search/tag/job/values when it should be api/v2/search/tag/resource.job/values. As a result, we always fallback to v1 of the API. In this case I believe that tags result are the exact same so this still works, however we should probably fix the issue with the API v2 tags query as we will more than likely eventually drop support for v1 and move to v2 + it removes one error from the network tab/console.

[fabrizio-grafana]

Should be fixed!

[joey-grafana]

This looks great! 👍