[ML] Context for recovered alerts

[darnautov]

Summary

Resolves #126803

Adds context for recovered alerts created by Anomaly Detection and Anomaly Detection Health rule types.

The recovered context is limited to the same context and the active alerts, hence there are no new fields.

For Anomaly detection recovered alerts contains job IDs and link to the Anomaly Explorer page.
For Anomaly detection health, the message field is updated according to the test and results contain extra data about executed tests.

How to test

Anomaly detection rule

1. Create a rule instance.
2. Attach an action that runs when "Recovered" (with a connector of your choice, e.g. Slack). At the moment framework doesn't allow to provide a default message for recovered alerts, so you can copy-paste the message from the "Anomaly score matched the condition" action group, it is

Default message example:

RECOVERED!
[{{rule.name}}] Elastic Stack Machine Learning Alert:
- Job IDs: {{context.jobIds}}
- Time: {{date}}

{{context.message}}

{{! Replace kibanaBaseUrl if not configured in Kibana }}
[Open in Anomaly Explorer]({{{kibanaBaseUrl}}}{{{context.anomalyExplorerUrl}}})

3. Alert is recovered when there is no conclusive anomaly found. Hence it's quite easy to test, just ingest some data to the source index that triggers an anomaly and wait for some time (depending on your job config, in particular, bucket_span and query_delay of the datafeed)

Note, that for recovered alerts only these fields from the context are populated (because there is actually no anomaly has been found):

1. context.jobIds
2. context.anomalyExplorerUrl
3. context.message

Anomaly detection health rule

The example with the "Datafeed is not started" test:

1. Create a rule instance with and assign some open jobs, make sure that "Datafeed is not started" test is enabled
2. Attach an action that runs when "Recovered" (with a connector of your choice, e.g. Slack). At the moment framework doesn't allow to provide a default message for recovered alerts, so you can copy-paste the message from the "Issue detected" action group, it is

Default message example:

RECOVERED!
[{{rule.name}}] Anomaly detection jobs health check result:
{{context.message}}
{{#context.results}}
Job ID: {{job_id}}
{{#datafeed_id}}Datafeed ID: {{datafeed_id}}
{{/datafeed_id}}{{#datafeed_state}}Datafeed state: {{datafeed_state}}
{{/datafeed_state}}{{#memory_status}}Memory status: {{memory_status}}
{{/memory_status}}{{#model_bytes}}Model size: {{model_bytes}}
{{/model_bytes}}{{#model_bytes_memory_limit}}Model memory limit: {{model_bytes_memory_limit}}
{{/model_bytes_memory_limit}}{{#peak_model_bytes}}Peak model bytes: {{peak_model_bytes}}
{{/peak_model_bytes}}{{#model_bytes_exceeded}}Model exceeded: {{model_bytes_exceeded}}
{{/model_bytes_exceeded}}{{#log_time}}Memory logging time: {{log_time}}
{{/log_time}}{{#failed_category_count}}Failed category count: {{failed_category_count}}
{{/failed_category_count}}{{#annotation}}Annotation: {{annotation}}
{{/annotation}}{{#missed_docs_count}}Number of missed documents: {{missed_docs_count}}
{{/missed_docs_count}}{{#end_timestamp}}Latest finalized bucket with missing docs: {{end_timestamp}}
{{/end_timestamp}}{{#errors}}Error message: {{message}} {{/errors}}
{{/context.results}}

5. Stop the datafeed of any anomaly detection job assigned to this rule. 6. Start the datafeed. You need to make sure that ALL jobs assigned to this rule have started datafeed. 7. Check the message, it should be something similar to

Checklist

• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[darnautov]

@elasticmachine merge upstream

[szabosteve]

Suggested change

	defaultMessage: `{count, plural, one {Job} other {Jobs}} {jobsString} reached the hard model memory limit. Assign the job more memory and restore from a snapshot from prior to reaching the hard limit.`,
	defaultMessage: `{count, plural, one {Job} other {Jobs}} {jobsString} reached the hard model memory limit. Assign more memory to the job and restore it from a snapshot taken prior to reaching the hard limit.`,

[darnautov]

FYI this text has been there for a while. It's also been reviewed before

[darnautov]

Updated in 552d005

[szabosteve]

Suggested change

	'{count, plural, one {Job} other {Jobs}} {jobsString} reached the soft model memory limit. Assign the job more memory or edit the datafeed filter to limit scope of analysis.',
	'{count, plural, one {Job} other {Jobs}} {jobsString} reached the soft model memory limit. Assign more memory to the job or edit the datafeed filter to limit the scope of analysis.',

[darnautov]

FYI this text has been there for a while. It's also been reviewed before

[szabosteve]

Thanks for the context. It still sounds better this way in my opinion. Please take it or leave it as you wish.

[darnautov]

Updated 552d005

[ymao1]

LGTM! Code review only for use of getRecoveredAlerts() inside the rule executors

[peteharverson]

Left another suggestion for one of the messages, but otherwise tested and LGTM.

[peteharverson]

I'd edit this to No anomalies have been found that exceed the severity threshold of {severity}.

I don't think it needs the square brackets around the severity value.

Or better still, could you include the lookback interval in there too? Such as

No anomalies have been found in the past {lookbackInterval} that exceed the severity threshold of {severity}.

[darnautov]

Updated in b27c03a

[szabosteve]

UI text LGTM.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 1776e02
• Storybooks Preview
• Webpack Bundle Analyzer

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
ml	3.3MB	3.3MB	+24.0B

History

• 💔 Build #46972 failed 552d005
• 💚 Build #46869 succeeded 171fc55
• 💔 Build #46840 failed af15a0d
• 💛 Build #46506 was flaky 721f38b
• 💔 Build #46426 failed fbee25b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @darnautov