[TSVB] Remove the broken "aggregate function" in TSVB table

[wylieconlon]

This is an issue originally reported on Discuss. If you go to TSVB > Table > Options > Aggregate function, it gives you form controls that configure a Series Agg, but the controls don't work.

This feature should be removed because it's confusing and redundant.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[wylieconlon]

@simianhacker @timroes Are you aware of this function ever working? Or is this change risk-free because it never worked?

[wylieconlon]

I just created a deployment using 7.0.1 and observed that the feature has existed since then, and the feature appears to be doing something but not necessarily something useful.

[simianhacker]

Yes, this worked when it was originally developed.

[wylieconlon]

@simianhacker can you clarify a bit more? When I tested it in 7.0 I wasn't sure what it was doing, since it seems like it's doing something different from what the series agg does.

To be more specific, the "Series agg" reduces the total number of visible series. It seems like the "aggregate function" is not capable of reducing the number of visible series.

[alexwizp]

@wylieconlon honestly, I think the initial idea of that was to do something like our new 'Entire Timerange mode' feature. But I'm not sure.

[simianhacker]

Here is the use case that worked at one point. Let's say your listing all your hosts and you want to have a column for network traffic. Keep in mind that network traffic for system.network.out.bytes is recorded for every interface which is identified by system.network.name. What you really want is OVERALL network traffic which is all the traffic aggregated together across all the interfaces. To do that you would setup the metrics tab (for a series) like this:

and the options tab like this

The query would throw a terms agg in there and then do something similar to the series_agg to aggregate all the values together for that column.

Here is the original PR: #12813

[stratoula]

Although, it doesn't seem to work as @simianhacker described I think we should first gather metrics on how many visualizations are using it and then decide if we want to remove it or fix it.

[flash1293]

Discussed offline, we decided to go with @stratoula s suggestion of implementing telemetry before removing it completely.

[simianhacker]

Please fix this feature and don't remove it, it's the only way to do any quasi-multiple-groupings of data.

[flash1293]

I traced this back - it broke in 7.2 during rollup integration in #28762 more specifically because of this change: https://github.com/elastic/kibana/pull/28762/files#diff-8356ba367c7c14f240fbaa3bcd6a3d4042c5d3e2a02193ffe2355ee5cda300dcR28 If i'm removing this part it works as expected.

In case of aggregate function in TSVB table, there's no timeseries agg directly in the pivot split, so it behaves as if it would be a rollup response. @alexwizp could you check whether there is an easy way to fix this?