From 2e2427de824a8192b32e5344f5a0204d61cd55c1 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Mon, 23 Dec 2024 10:31:44 -0500 Subject: [PATCH 1/2] [Serverless][8.18] EQL Sequence alert suppression (#6291) * First draft * draft 1 * Update docs/detections/alert-suppression.asciidoc * fix it? * Moves info * updating ref * Update docs/detections/building-block-rule.asciidoc * Update docs/serverless/rules/building-block-rule.asciidoc * Removing empty lines * Removes tech preview label for 8.18 * updates note about reqs * Re-adds + * Fixes Serverless note * Fixes numebring (cherry picked from commit ed389cefbf7189f9df75e3f3b02f15c3cce0272a) # Conflicts: # docs/serverless/alerts/alert-suppression.asciidoc --- docs/detections/alert-suppression.asciidoc | 10 +- docs/detections/rules-ui-create.asciidoc | 3 +- .../alerts/alert-suppression.asciidoc | 134 ++++++++++++++++++ 3 files changed, 139 insertions(+), 8 deletions(-) create mode 100644 docs/serverless/alerts/alert-suppression.asciidoc diff --git a/docs/detections/alert-suppression.asciidoc b/docs/detections/alert-suppression.asciidoc index 9cc392cd38..320d74ef18 100644 --- a/docs/detections/alert-suppression.asciidoc +++ b/docs/detections/alert-suppression.asciidoc @@ -7,8 +7,6 @@ * Alert suppression requires a https://www.elastic.co/pricing[Platinum or higher subscription]. * {ml-cap} rules have <> for alert suppression. - -preview::["Alert suppression is in technical preview for event correlation rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."] -- Alert suppression allows you to reduce the number of repeated or duplicate detection alerts created by these detection rule types: @@ -16,7 +14,7 @@ Alert suppression allows you to reduce the number of repeated or duplicate detec * <> * <> * <> -* <> (non-sequence queries only) +* <> * <> * <> * <> @@ -34,7 +32,7 @@ You can configure alert suppression when you create or edit a supported rule typ . When configuring the rule type (the *Define rule* step for a new rule, or the *Definition* tab for an existing rule), specify how you want to group events for alert suppression: + -- -* **Custom query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules:** In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. +* **Custom query, indicator match, threshold, event correlation, new terms, {ml}, and {esql} rules:** In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. * **Threshold rule:** In *Group by*, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. -- @@ -45,7 +43,7 @@ If you specify a field with multiple values, alerts with that field are handled * **Custom query or threshold rules:** A group of alerts is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. * **Indicator match, event correlation (non-sequence queries only), new terms, {esql}, or {ml} rules:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. - +* **Event correlation (sequence queries only) rules:** If the specified field contains an array of values, suppression only happens if the field's values are an exact match and in the same order. For example, if you specify the field `myips` and one sequence alert has [1.1.1.1, 0.0.0.0] and another sequence alert has [1.1.1.1, 192.168.0.1], neither of those alerts will be suppressed, despite sharing an array element. ====== . If available, select how often to create alerts for duplicate events: @@ -114,5 +112,5 @@ image::images/timeline-button.png[Investigate in timeline button, 200] Some rule types have a maximum number of alerts that can be suppressed (custom query rules don't have a suppression limit): -* **Threshold, event correlation (non-sequence queries only), {esql}, and {ml}:** The maximum number of alerts is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. +* **Threshold, event correlation, {esql}, and {ml}:** The maximum number of alerts is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. * **Indicator match and new terms:** The maximum number is five times the value you choose for the rule's **Max alerts per run** <>. The default value is `100`, which means the default maximum limit for indicator match rules and new term rules is `500`. \ No newline at end of file diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 1638a6664c..9491f98c19 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -205,9 +205,8 @@ NOTE: For sequence events, the {security-app} generates a single alert when all * *Timestamp field*: Contains the event timestamp used for sorting a sequence of events. This is different from the *Timestamp override* advanced setting, which is used for querying events within a range. Defaults to the `@timestamp` ECS field. + -. preview:[] (Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Use *Suppress alerts by* to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. +. Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Use *Suppress alerts by* to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. + - //// The following steps are repeated across multiple rule types. If you change anything in these steps or sub-steps, apply the change to the other rule types, too. diff --git a/docs/serverless/alerts/alert-suppression.asciidoc b/docs/serverless/alerts/alert-suppression.asciidoc new file mode 100644 index 0000000000..51650665d6 --- /dev/null +++ b/docs/serverless/alerts/alert-suppression.asciidoc @@ -0,0 +1,134 @@ +[[security-alert-suppression]] += Suppress detection alerts + +// :description: Reduce noise from rules that create repeated or duplicate alerts. +// :keywords: serverless, security, how-to + +++++ +Suppress alerts +++++ + + +.Requirements and notice +[IMPORTANT] +==== +* {ml-cap} rules have <> for alert suppression. +* Alert suppression is in technical preview for event correlation rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. +==== + +Alert suppression allows you to reduce the number of repeated or duplicate detection alerts created by these detection rule types: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> + +Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values. + +The {security-app} displays several indicators in the Alerts table and the alert details flyout when a detection alert is created with alert suppression enabled. You can view the original events associated with suppressed alerts by investigating the alert in Timeline. + +[NOTE] +==== +Alert suppression is not available for Elastic prebuilt rules. However, if you want to suppress alerts for a prebuilt rule, you can duplicate it, then configure alert suppression on the duplicated rule. +==== + +[discrete] +[[security-alert-suppression-configure-alert-suppression]] +== Configure alert suppression + +You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating <>, <>, <>, <>, <>, <>, or <> rules for detailed instructions. + +. When configuring the rule type (the **Define rule** step for a new rule, or the **Definition** tab for an existing rule), specify how you want to group events for alert suppression: ++ +** **Custom query rule, indicator match, threshold, event correlation, new terms, {esql}, or {ml} rules:** In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. +** **Threshold rule:** In **Group by**, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. ++ +[NOTE] +==== +If you specify a field with multiple values, alerts with that field are handled as follows: + +* **Custom query or threshold rules:** Alerts are grouped by each unique value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. +* **Indicator match, event correlation (non-sequence queries only), new terms, {esql}, or {ml} rules:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. +* **Event correlation (sequence queries only) rules:** If the suppression field is an array of values, the suppressed alert will only suppress values that are an exact match. The values must be equivalent and be in the same position. For example, if you configure suppresson on the field `myips` and one sequence alert has [1.1.1.1, 0.0.0.0] and another sequence alert has [1.1.1.1, 192.168.0.1], neither of those alerts will be suppressed, despite sharing an array element. +==== +. If available, select how often to create alerts for duplicate events: ++ +.NOTE +[NOTE] +==== +Both options are available for custom query, indicator match, event correlation, new terms, {esql}, and {ml} rules. Threshold rules only have the **Per time period** option. +==== ++ +** **Per rule execution**: Create an alert each time the rule runs and an event meets its criteria. +** **Per time period**: Create one alert for all qualifying events that occur within a specified time window, beginning from when an event first meets the rule criteria and creates the alert. ++ +For example, if a rule runs every 5 minutes but you don't need alerts that frequently, you can set the suppression time period to a longer time, such as 1 hour. If the rule meets its criteria, it creates an alert at that time, and for the next hour, it'll suppress any subsequent qualifying events. ++ +[role="screenshot"] +image:images/alert-suppression/-detections-alert-suppression-options.png[Alert suppression options] +. Under **If a suppression field is missing**, choose how to handle events with missing suppression fields (events in which one or more of the **Suppress alerts by** fields don't exist): ++ +.NOTE +[NOTE] +==== +These options are not available for threshold rules. +==== ++ +** **Suppress and group alerts for events with missing fields**: Create one alert for each group of events with missing fields. Missing fields get a `null` value, which is used to group and suppress alerts. +** **Do not suppress alerts for events with missing fields**: Create a separate alert for each matching event. This basically falls back to normal alert creation for events with missing suppression fields. +. Configure other rule settings, then save and enable the rule. + +.Tips +[TIP] +==== +* Use the **Rule preview** before saving the rule to visualize how alert suppression will affect the alerts created, based on historical data. +* If a rule times out while suppression is turned on, try shortening the rule's <> time or turn off suppression to improve the rule's performance. +==== + +[discrete] +[[security-alert-suppression-confirm-suppressed-alerts]] +== Confirm suppressed alerts + +The {security-app} displays several indicators of whether a detection alert was created with alert suppression enabled, and how many duplicate alerts were suppressed. + +[IMPORTANT] +==== +After an alert is moved to the `Closed` status, it will no longer suppress new alerts. To prevent interruptions or unexpected changes in suppression, avoid closing alerts before the suppression interval ends. +==== + +* **Alerts** table — Icon in the **Rule** column. Hover to display the number of suppressed alerts: ++ +[role="screenshot"] +image:images/alert-suppression/-detections-suppressed-alerts-table.png[Suppressed alerts icon and tooltip in Alerts table] +* **Alerts** table — Column for suppressed alerts count. Select **Fields** to open the fields browser, then add `kibana.alert.suppression.docs_count` to the table. ++ +[role="screenshot"] +image:images/alert-suppression/-detections-suppressed-alerts-table-column.png[Suppressed alerts count field column in Alerts table] +* Alert details flyout — **Insights** section: ++ +[role="screenshot"] +image:images/alert-suppression/-detections-suppressed-alerts-details.png[Suppressed alerts Insights section in alert details flyout] + +[discrete] +[[security-alert-suppression-investigate-events-for-suppressed-alerts]] +== Investigate events for suppressed alerts + +With alert suppression, detection alerts aren't created for the grouped source events, but you can still retrieve the events for further analysis or investigation. Do one of the following to open Timeline with the original events associated with both the created alert and the suppressed alerts: + +* **Alerts** table — Select **Investigate in timeline** in the **Actions** column. ++ +[role="screenshot"] +image:images/alert-suppression/-detections-timeline-button.png[Investigate in timeline button] +* Alert details flyout — Select **Take action** → **Investigate in timeline**. + +[discrete] +[[security-alert-suppression-alert-suppression-limit-by-rule-type]] +== Alert suppression limit by rule type + +Some rule types have a maximum number of alerts that can be suppressed (custom query rules don't have a suppression limit): + +* **Threshold, event correlation, {esql}, and {ml}:** The maximum number is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. +* **Indicator match and new terms:** The maximum number is five times the value you choose for the rule's **Max alerts per run** <>. The default value is `100`, which means the default maximum limit for indicator match rules and new terms rules is `500`. From 186494786e935967ace0757e3967f40181723be6 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 23 Dec 2024 15:32:57 +0000 Subject: [PATCH 2/2] Delete docs/serverless directory and its contents --- .../alerts/alert-suppression.asciidoc | 134 ------------------ 1 file changed, 134 deletions(-) delete mode 100644 docs/serverless/alerts/alert-suppression.asciidoc diff --git a/docs/serverless/alerts/alert-suppression.asciidoc b/docs/serverless/alerts/alert-suppression.asciidoc deleted file mode 100644 index 51650665d6..0000000000 --- a/docs/serverless/alerts/alert-suppression.asciidoc +++ /dev/null @@ -1,134 +0,0 @@ -[[security-alert-suppression]] -= Suppress detection alerts - -// :description: Reduce noise from rules that create repeated or duplicate alerts. -// :keywords: serverless, security, how-to - -++++ -Suppress alerts -++++ - - -.Requirements and notice -[IMPORTANT] -==== -* {ml-cap} rules have <> for alert suppression. -* Alert suppression is in technical preview for event correlation rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. -==== - -Alert suppression allows you to reduce the number of repeated or duplicate detection alerts created by these detection rule types: - -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values. - -The {security-app} displays several indicators in the Alerts table and the alert details flyout when a detection alert is created with alert suppression enabled. You can view the original events associated with suppressed alerts by investigating the alert in Timeline. - -[NOTE] -==== -Alert suppression is not available for Elastic prebuilt rules. However, if you want to suppress alerts for a prebuilt rule, you can duplicate it, then configure alert suppression on the duplicated rule. -==== - -[discrete] -[[security-alert-suppression-configure-alert-suppression]] -== Configure alert suppression - -You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating <>, <>, <>, <>, <>, <>, or <> rules for detailed instructions. - -. When configuring the rule type (the **Define rule** step for a new rule, or the **Definition** tab for an existing rule), specify how you want to group events for alert suppression: -+ -** **Custom query rule, indicator match, threshold, event correlation, new terms, {esql}, or {ml} rules:** In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. -** **Threshold rule:** In **Group by**, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. -+ -[NOTE] -==== -If you specify a field with multiple values, alerts with that field are handled as follows: - -* **Custom query or threshold rules:** Alerts are grouped by each unique value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. -* **Indicator match, event correlation (non-sequence queries only), new terms, {esql}, or {ml} rules:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. -* **Event correlation (sequence queries only) rules:** If the suppression field is an array of values, the suppressed alert will only suppress values that are an exact match. The values must be equivalent and be in the same position. For example, if you configure suppresson on the field `myips` and one sequence alert has [1.1.1.1, 0.0.0.0] and another sequence alert has [1.1.1.1, 192.168.0.1], neither of those alerts will be suppressed, despite sharing an array element. -==== -. If available, select how often to create alerts for duplicate events: -+ -.NOTE -[NOTE] -==== -Both options are available for custom query, indicator match, event correlation, new terms, {esql}, and {ml} rules. Threshold rules only have the **Per time period** option. -==== -+ -** **Per rule execution**: Create an alert each time the rule runs and an event meets its criteria. -** **Per time period**: Create one alert for all qualifying events that occur within a specified time window, beginning from when an event first meets the rule criteria and creates the alert. -+ -For example, if a rule runs every 5 minutes but you don't need alerts that frequently, you can set the suppression time period to a longer time, such as 1 hour. If the rule meets its criteria, it creates an alert at that time, and for the next hour, it'll suppress any subsequent qualifying events. -+ -[role="screenshot"] -image:images/alert-suppression/-detections-alert-suppression-options.png[Alert suppression options] -. Under **If a suppression field is missing**, choose how to handle events with missing suppression fields (events in which one or more of the **Suppress alerts by** fields don't exist): -+ -.NOTE -[NOTE] -==== -These options are not available for threshold rules. -==== -+ -** **Suppress and group alerts for events with missing fields**: Create one alert for each group of events with missing fields. Missing fields get a `null` value, which is used to group and suppress alerts. -** **Do not suppress alerts for events with missing fields**: Create a separate alert for each matching event. This basically falls back to normal alert creation for events with missing suppression fields. -. Configure other rule settings, then save and enable the rule. - -.Tips -[TIP] -==== -* Use the **Rule preview** before saving the rule to visualize how alert suppression will affect the alerts created, based on historical data. -* If a rule times out while suppression is turned on, try shortening the rule's <> time or turn off suppression to improve the rule's performance. -==== - -[discrete] -[[security-alert-suppression-confirm-suppressed-alerts]] -== Confirm suppressed alerts - -The {security-app} displays several indicators of whether a detection alert was created with alert suppression enabled, and how many duplicate alerts were suppressed. - -[IMPORTANT] -==== -After an alert is moved to the `Closed` status, it will no longer suppress new alerts. To prevent interruptions or unexpected changes in suppression, avoid closing alerts before the suppression interval ends. -==== - -* **Alerts** table — Icon in the **Rule** column. Hover to display the number of suppressed alerts: -+ -[role="screenshot"] -image:images/alert-suppression/-detections-suppressed-alerts-table.png[Suppressed alerts icon and tooltip in Alerts table] -* **Alerts** table — Column for suppressed alerts count. Select **Fields** to open the fields browser, then add `kibana.alert.suppression.docs_count` to the table. -+ -[role="screenshot"] -image:images/alert-suppression/-detections-suppressed-alerts-table-column.png[Suppressed alerts count field column in Alerts table] -* Alert details flyout — **Insights** section: -+ -[role="screenshot"] -image:images/alert-suppression/-detections-suppressed-alerts-details.png[Suppressed alerts Insights section in alert details flyout] - -[discrete] -[[security-alert-suppression-investigate-events-for-suppressed-alerts]] -== Investigate events for suppressed alerts - -With alert suppression, detection alerts aren't created for the grouped source events, but you can still retrieve the events for further analysis or investigation. Do one of the following to open Timeline with the original events associated with both the created alert and the suppressed alerts: - -* **Alerts** table — Select **Investigate in timeline** in the **Actions** column. -+ -[role="screenshot"] -image:images/alert-suppression/-detections-timeline-button.png[Investigate in timeline button] -* Alert details flyout — Select **Take action** → **Investigate in timeline**. - -[discrete] -[[security-alert-suppression-alert-suppression-limit-by-rule-type]] -== Alert suppression limit by rule type - -Some rule types have a maximum number of alerts that can be suppressed (custom query rules don't have a suppression limit): - -* **Threshold, event correlation, {esql}, and {ml}:** The maximum number is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. -* **Indicator match and new terms:** The maximum number is five times the value you choose for the rule's **Max alerts per run** <>. The default value is `100`, which means the default maximum limit for indicator match rules and new terms rules is `500`.