[APM] Automatically select right resolution for transaction metrics

[dgieselaar]

We are planning to continuously rollup metrics in different resolutions, to improve query performance for bigger time ranges [1]. When querying transaction metrics, we should automatically select the right resolution to query by executing a pre-flight requests that determines the appropriate resolution, and whether data is available for that resolution.

The easiest way to do this is to change the implementation of getSearchTransactionsEvents to return a resolution instead of a boolean. We can also create a new function and incrementally replace usage of getSearchTransactionsEvents with the new function. If we go for a new function, it should ideally be fetched in TimeRangeMetadataContext.

[1] https://github.com/elastic/apm-dev/issues/877 (internal)

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)

[felixbarny]

How do we handle the case where we only have only partial 1m data for the selected time range? Will we show data in 1m resolution but have missing data at the beginning of the time range? Or will we fall back to 10m resolution if it is retained longer than the 1m resolution?

[dgieselaar]

if there's any data for given time frame for the preferred resolution, we will use the preferred resolution.

[graphaelli]

Can you clarify whether this only covers the curated UI or also our bundled alerts?

[sorenlouv]

Can you clarify whether this only covers the curated UI or also our bundled alerts?

My impression is that what Dario said is also the case for rules (created by APM UI). So if any data is found for the preferred resolution the rule will only take that into account when determining if it should trigger an alert.

We recently added a warning to rules that users should not decrease the lookback window to less than 5 minutes. This makes them more reliable when using transaction metrics where data will be at least 1 minute delayed and possibly more (network congestion, queueing etc).

If we want to support shorter lookback windows (less than 5 minutes) we may need to use transaction events instead of metrics in these cases - although that won't scale well.

[dgieselaar]

@graphaelli @sqren I've opened #148507 to capture the work needed to make sure the UI, alerts and ML don't break.

[felixbarny]

Is this done or is there something missing?

[dgieselaar]

@felixbarny We've only migrated the service inventory, not other views that only use transaction metrics.

[felixbarny]

ah, right

[sorenlouv]

@dgieselaar This is marked as "In Progress" and assigned to you. Is that still the case or should this be moved to the backlog?

[dgieselaar]

@sqren yeah, it's a bit of an ongoing project, not everything that can be moved to rollups has moved to rollups - we should plan some additional work for the upcoming cycles if we have capacity. Alternatively, we can close this and put in a pitch for a future release.

[felixbarny]

Can we have a meta-issue that tracks the changes that we've done so far and the ones that are pending?

[sorenlouv]

Okay, closing this. I've created a meta issue for the issues we've closed and are currently working on: #157227