TimeSeries: Explicitly add transformer when timeseries-long exists

[ryantxu]

This is revisiting the auto-magic transformation added in #62732

While nice, the result is that many subtle features do not work as expected because the field names do not exist for overrides and customization. The legend toggles also do not work.

What is this feature?

This replaces the automatic transform with a button that will add the transform explicitly.

Why do we need this feature?

Having explicit control of the shape will cause fewer surprises long term

TODO:

• transformation tests... @bohandley ?
• devenv dashboard
• remove/update Dataplane: Support timeSeriesLong without transform #62732 changelog info for 9.5?

#63901 will need something similar

[torkelo]

Good catch, it think we have to do this, a transformation that adds/changes field names cannot be done inside the visualization.

One possible alternative would be that each panel plugin could also expose a transformation function that is called as part of the data processing (before applyFieldOverrides), but that could be really messy to implement (both in current arch and scenes) as data production (query runners) are pretty separate from visualizations. However there is actually link in the scenes framework (the VizPanel checks if there is a query runner on its own level and sets the MaxDataPoints based on viz width) , this could in theory also add a transformation callback specified in the plugin

[ryantxu]

🤔 -- long term, perhaps plugins could specific the set of DataFrameTypes it supports, and the PanelQueryRunner (or scenes) could insert a step to convert before calling overrides.

But I think this is a reasonable approach to get us there explicitly. The downside is when flipping panel types, the xform may not be ncessary. But i think that is a small price to pay for more consistency

[kylebrandt]

@ryantxu can you expand a bit more on why this doesn't (or won't) work? Would like to understand why inserting the transformation is needed.

[leeoniya]

a possible issue here is that not all string fields may be relevant for partitioning. for instance, fields used for datalinks. e.g. #64088

the partitionByValues transformer UI offers a field selection so some fields can be opted out of, but the Prepare time series UI does not.

[ryantxu]

understand why inserting the transformation is needed.

If we are only using default settings, it can work fine -- however if the panel requires any configuration/field(series) then the infrastructure to assign that config happens before we run the long-to-wide conversion. This means odd things do not work like:

• toggling visibility in the legend
• applying field overrides
• anywhere we have a field picker UI element

In #62732 @bohandley applies the transformation inside the panel, but this PR moves it so it will happen before the panel gets the data.

[bohandley]

understand why inserting the transformation is needed.

If we are only using default settings, it can work fine -- however if the panel requires any configuration/field(series) then the infrastructure to assign that config happens before we run the long-to-wide conversion. This means odd things do not work like:

• toggling visibility in the legend
• applying field overrides
• anywhere we have a field picker UI element

In #62732 @bohandley applies the transformation inside the panel, but this PR moves it so it will happen before the panel gets the data.

I like this! If we add it early enough, then the panel will have everything it needs and we won't have to explicitly bring in the transformation, right? I think if we are going to support time series long, we should do it implicitly, with no extra clicks. This is possible with the right amount of checks, right? @ryantxu @leeoniya @kylebrandt I remember my first try was way outside the panel and I think it might have worked. I can try that out again.

[grafanabot]

This pull request was removed from the 9.5.0 milestone because 9.5.0 is currently being released.

[torkelo]

@ryantxu @bohandley we should revert #62732 before 9.5 stable is released

[bohandley]

@ryantxu @bohandley we should revert #62732 before 9.5 stable is released

@torkelo I just looked over reverting #62732 and tried to revert it remembering these changes have already been fixed. #62732 made changes in app/plugins/panel/timeseries/utils.ts and these changes were removed in this PR. The new changes move the functionality into the internal logic of the time series tranformer so that the transform can be applied explicitly by choosing to do so instead of automatically. This fixes the issue so the PR #62732 does not need to be reverted. Thanks @ryantxu!

[ryantxu]

@bohandley / @kylebrandt -- I think this is good to go now.