Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed explore-analyze/images/ml-anomaly-alert.jpg
Binary file not shown.
Binary file added explore-analyze/images/ml-anomaly-alert.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -56,19 +56,32 @@ To set up an {{anomaly-detect}} alert rule:
The `anomaly_score` indicates the significance of a given anomaly compared to
previous anomalies. The default severity threshold is 75 which means every
anomaly with an `anomaly_score` of 75 or higher triggers the associated action.
6. Select whether you want to include interim results. Interim results are created before a bucket is finalized and might disappear after full processing.
6. {applies_to}`stack: ga 9.3`{applies_to}`serverless: ga` (Optional) To narrow down the list of anomalies that the rule looks for, add an **Anomaly filter**. This feature uses KQL and is only available for the Record and Influencer result types.

In the **Anomaly filter** field, enter a KQL query that specifies conditions to alert on. You can set up the following conditions:

* One or more partitioning or influencers fields in the anomaly results match the specified conditions
* The actual or typical scores in the anomalies match the specified conditions

For example, say you've set up alerting for an anomaly detection job that has `partition_field = "response.keyword"` as the detector. If you were only interested in being alerted on `response.keyword = 404`, enter `partition_field_value: "404"` into the **Anomaly filter** field. When the rule runs, it will only alert on anomalies with `partition_field_value: "404"`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice example!


::::{note}
When creating the KQL query, you're given suggestions for the most relevant fields to filter by. To compare actual and typical values, use operators such as `>` (greater than), `<` (less than), or `=` (equal to).
::::

7. Select whether you want to include interim results. Interim results are created before a bucket is finalized and might disappear after full processing.
- Include interim results if you
want to be notified earlier about a potential anomaly even if it might be a
false positive.
- Don't include interim results if you want to get notified only about anomalies of fully
processed buckets.

:::{image} /explore-analyze/images/ml-anomaly-alert.jpg
:::{image} /explore-analyze/images/ml-anomaly-alert.png
:alt: Selecting result type, severity, and interim results
:screenshot:
:::
Comment on lines +79 to 82
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whereas the previous screenshot felt too obvious to be needed, this one feels more confusing than helpful. I had to re-read step seven a few times before I saw the Include interim results toggle at the bottom.

Is this necessary? Why does step 7 get a screenshot when step 4 (Select a type of machine learning result. You can create rules based on bucket, record, or influencer results.), for example, does not?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also very similar to solutions/images/serverless-anomaly-detection-alert.png, but different enough to mean we need to update two screenshots instead of one when they inevitably go stale.


7. (Optional) Configure **Advanced settings**:
8. (Optional) Configure **Advanced settings**:
- Configure the _Lookback interval_ to define how far back to query previous anomalies during each condition check. Its value is derived from the bucket span of the job and the query delay of the {{dfeed}} by default. It is not recommended to set the lookback interval lower than the default value, as it might result in missed anomalies.
- Configure the _Number of latest buckets_ to specify how many buckets to check to obtain the highest anomaly score found during the _Lookback interval_. The alert is created based on the highest scoring anomaly from the most anomalous bucket.

Expand All @@ -86,8 +99,8 @@ You can preview how the rule would perform on existing data:
:screenshot:
:::

8. Set how often to check the rule conditions by selecting a time value and unit under **Rule schedule**.
9. (Optional) Configure **Advanced options**:
9. Set how often to check the rule conditions by selecting a time value and unit under **Rule schedule**.
10. (Optional) Configure **Advanced options**:
- Define the number of consecutive matches required before an alert is triggered under **Alert delay**.
- Enable or disable **Flapping Detection** to reduce noise from frequently changing alerts. You can customize the flapping detection settings if you need different thresholds for detecting flapping behavior.

Expand Down
Binary file modified solutions/images/serverless-anomaly-detection-alert.png
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another interesting one. Of the three screenshots I've commented on, this is the one you could probably easiest convince me to stay. But that's mostly because it's pretty; It shows off the feature config which looks quite nice.

I'm curious how we use screenshots elsewhere. Is this the typical pattern? What I mean by that is that this screenshot is in step 5, but then applies to steps 6-14.

Admittedly, this is not my area of expertise. Would love to hear your thoughts and @florent-leborgne's.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading this too but there's not much to go off of: https://docs.elastic.dev/tech-writing-guidelines/ui-writing#screenshots.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -50,18 +50,32 @@ To create an anomaly detection rule:
| **Influencer** | The most unusual entities in a time range |

7. Adjust the **Severity** to match the anomaly score that will trigger the action. The anomaly score indicates the significance of a given anomaly compared to previous anomalies. The default severity threshold is 75, which means every anomaly with an anomaly score of 75 or higher will trigger the associated action.
8. (Optional) Turn on **Include interim results** to include results that are created by the anomaly detection job *before* a bucket is finalized. These results might disappear after the bucket is fully processed. Include interim results if you want to be notified earlier about a potential anomaly even if it might be a false positive.
9. (Optional) Expand and change **Advanced settings**:
8. {applies_to}`stack: ga 9.3`{applies_to}`serverless: ga` (Optional) To narrow down the list of anomalies that the rule monitors for, add an **Anomaly filter**. This filtering capability uses KQL and is only available for the Record and Influencer result types.

In the **Anomaly filter** field, enter a KQL query that specifies to only alert when either or both happen:

* Certain partitioning or influencers fields in the anomaly results match specified conditions
* The actual or typical scores in the anomalies match specified conditions

For example, say you've set up alerting for an anomaly detection job that has `partition_field = "response.keyword"` as the detector. If you were only interested in being alerted on `response.keyword = 404`, enter `partition_field_value: "404"` into the **Anomaly filter** field. When the rule runs, it will only alert on anomalies with `partition_field_value: "404"`.

::::{note}
When creating the KQL query, you're given suggestions for the most relevant fields to filter by. To compare actual and typical values, use operators such as `>` (greater than), `<` (less than), or `=` (equal to).
::::

9. (Optional) Turn on **Include interim results** to include results that are created by the anomaly detection job *before* a bucket is finalized. These results might disappear after the bucket is fully processed. Include interim results if you want to be notified earlier about a potential anomaly even if it might be a false positive.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
9. (Optional) Turn on **Include interim results** to include results that are created by the anomaly detection job *before* a bucket is finalized. These results might disappear after the bucket is fully processed. Include interim results if you want to be notified earlier about a potential anomaly even if it might be a false positive.
9. (Optional) Turn on **Include interim results** to include results created by the anomaly detection job *before* their bucket is finalized. These results might disappear after the bucket is fully processed. Include interim results if you want to be notified earlier about a potential anomaly even if it might be a false positive.


10. (Optional) Expand and change **Advanced settings**:

| Setting | Description |
| --- | --- |
| **Lookback interval** | The interval used to query previous anomalies during each condition check. Setting the lookback interval lower than the default value might result in missed anomalies. |
| **Number of latest buckets** | The number of buckets to check to obtain the highest anomaly from all the anomalies that are found during the Lookback interval. An alert is created based on the anomaly with the highest anomaly score from the most anomalous bucket. |

10. (Optional) Under **Check the rule condition with an interval**, specify an interval, then click **Test** to check the rule condition with the interval specified. The button is grayed out if the datafeed is not started. To test the rule, start the data feed.
11. (Optional) If you want to change how often the condition is evaluated, adjust the **Check every** setting.
12. (Optional) Set up **Actions**.
13. **Save** your rule.
11. (Optional) Under **Check the rule condition with an interval**, specify an interval, then click **Test** to check the rule condition with the interval specified. The button is grayed out if the datafeed is not started. To test the rule, start the data feed.
12. (Optional) If you want to change how often the condition is evaluated, adjust the **Check every** setting.
13. (Optional) Set up **Actions**.
14. **Save** your rule.

::::{note}
Anomaly detection rules are defined as part of a job. Alerts generated by these rules do not appear on the **Alerts** page.
Expand Down
Loading