[Logs UI] Adapt log entry rate data visualisations to new designs

[Kerry350]

⚠️ ⚠️ ⚠️ There are some smaller discussion points still outstanding (see below) ⚠️ ⚠️ ⚠️

Summary

The log entry rate visualisation should be adapted to match the new designs.

With the new designs we will have two distinct sections: 1) Log entry rate (Logs entries) 2) Anomalies

Log entry rate section

The log entry rate section will contain a stacked bar chart that visualises the log entry rate. Each bar represents a bucket, and each stacked portion of the bar represents a data set (from event.dataset). There will be a legend, to toggle the visibility of datasets.

Anomalies section

The anomalies section will contain a leading chart at the top of the section. This leading chart will again contain a bar chart (in grey) that shows the overall log entry rate, however the bars will not be stacked based on dataset here. The annotation overlays will show the areas where there are anomalies. These should only be displayed for major (yellow) and critical (red) anomalies (based on severity score).

Underneath the leading chart there will be a table detailing the top (based on severity score) n (5?) datasets with anomalies. These rows will be collapsed by default, clicking the toggle will open the section and display a chart specific to the dataset. This chart will be a filtered version of the leading chart (overall log entry rate and anomaly annotations but specific to the dataset).

For these charts there are some stats displayed to the right. And also the "Analyze in ML" buttons which should be implemented in #46445.

Pending discussion points

Can be seen in the annotations on this image:

Acceptance criteria

• There will be two sections "Logs entries" and "Annotations"
• The logs entries chart will contain a stacked bar chart detailing overall log entry rate, stacked based on the dataset. It will have a legend.
• The anomalies section will have an overall leading chart with an overall log entry rate bar chart, and annotations for anomalies
• The anomalies section will have a table for the datasets with the most anomalies, the rows can be toggled to display a chart that shows the overall log entry rate and anomalies for that specific dataset
• Anomalies charts should show the relevant stats to the right

[elasticmachine]

Pinging @elastic/infra-logs-ui (Team:infra-logs-ui)

[sophiec20]

Adding some clarification about the data being used in the charts..

The anomaly results may have a different time span to the source data for the following reasons

• if log data is archived, anomaly results will still exist for historical data that was analyzed
• during job start up, anomaly detection can be configured to lookback over all historical data or perhaps only over last 2 weeks (likely default). Therefore, during the initial job start up period, the span of anomaly results will not match the span of source data (rough estimate that lookback could take ~1m for medium sized log data). Also, the span of the anomaly results may not go back as far in time if the lookback was configured for last x weeks.
• older anomaly results may have been deleted, which is possible but unlikely.

The top Log Entries chart could either be plotted from source data or from the ML anomaly results.

If source data, then the span of data will be according to the log archive policy being used. Which would look strange if very different from the span of the anomaly results. However it would be the only data source in environments where ML jobs had not yet been set up and would be a useful visual for anyone not (yet) using ML.

If ML results then the span of the data will match the anomaly charts below and use the same bucket time summarisation (15m configurable in job config), so the in-page experience is coherent. (However in the case of delayed ingest, where data arrives with too much of a lag to have been analyzed by ML, then the source data would be different. This is documented in ML and the job can be configured to allow for longer ingest lags and warns when data is missing.)

From the recent call, I think the plan was to plot from the ml results, so below proposes data sources for each chart with this assumption.

// Log entries - top chart

• event rate: plot sum(actual) where result_type: model_plot AND partition_field_name: event.dataset AND job_id: xxx and split by partition_field_value

// anomalies chart - Overall

• event rate: plot sum(actual) where result_type: model_plot AND job_id: xxx (although could also be event_count where result_type: bucket however best keep aligned with the dataset specific chart)
• anomalies: plot max(anomaly_score) where result_type: bucket AND job_id: xxx

// event.dataset specific chart

• event rate: plot sum(actual) where result_type: model_plot AND partition_field_value: yyy AND partition_field_name: event.dataset AND job_id: xxx
• anomalies: plot max(record_score) where result_type: record AND partition_field_value: yyy AND partition_field_name: event.dataset AND job_id: xxx

We also discussed only plotting scores above 50. With hindsight, I think it would be worth showing all anomalies; critical >=75, major >=50, minor >=25, warning >=3, (low >= 0 optional) -- as this matches with the ML app experience and could still be hidden using the charting component.

cc @grabowskit @stevedodson @peteharverson

[Kerry350]

@sophiec20 Thank you for this detailed response 👍

From the recent call, I think the plan was to plot from the ml results, so below proposes data sources for each chart with this assumption.

That’s correct - everything here is based on ML results, no raw source data at all. I imagine we’ll look to add that in 7.6 (or later). Firstly, so as you say, it’s useful to everyone and secondly we can show the benefits of ML.

Those data source examples are useful, thank you. We made the amendments to our log entry rate results API in #46751 (which provides the data powering these). It looks like our queries closely reflect what you’ve provided (maybe some minor tweaks needed).

We also discussed only plotting scores above 50. With hindsight, I think it would be worth showing all anomalies; critical >=75, major >=50, minor >=25, warning >=3, (low >= 0 optional) -- as this matches with the ML app experience and could still be hidden using the charting component.

Makes sense 👍