From c129aed519ea682aa835617fa86f70b67a671469 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Thu, 19 Sep 2024 00:13:49 -0400 Subject: [PATCH 1/9] First draft --- docs/getting-started/advanced-setting.asciidoc | 8 ++++++++ docs/serverless/settings/advanced-settings.mdx | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/docs/getting-started/advanced-setting.asciidoc b/docs/getting-started/advanced-setting.asciidoc index 2afc093217..9911a0c8cf 100644 --- a/docs/getting-started/advanced-setting.asciidoc +++ b/docs/getting-started/advanced-setting.asciidoc @@ -176,3 +176,11 @@ By default, Elastic prebuilt rules in the *Rules* and *Rule Monitoring* tables i == Manage alert tag options The `securitySolution:alertTags` field determines which options display in the alert tag menu. The default alert tag options are `Duplicate`, `False Positive`, and `Further investigation required`. You can update the alert tag menu by editing these options or adding more. To learn more about using alert tags, refer to <>. + +[discrete] +[[exclude-cold-frozen-data-rule-executions]] +== Exclude cold and frozen data tiers from rule executions + +Set the `excludedDataTiersForRuleExecution` field to ensure that events from cold and frozen {ref}/data-tiers.html[data tiers] aren't searched during rule executions. Excluding data from cold and frozen tiers might help to improve rule performance or reduce execution time. + +If you specify multiple data tiers, values must be separated by commas, for example: `data_frozen`, `data_cold`. This setting is turned off by default. \ No newline at end of file diff --git a/docs/serverless/settings/advanced-settings.mdx b/docs/serverless/settings/advanced-settings.mdx index 67046cb80a..f8b8cd7117 100644 --- a/docs/serverless/settings/advanced-settings.mdx +++ b/docs/serverless/settings/advanced-settings.mdx @@ -196,3 +196,11 @@ By default, Elastic prebuilt rules in the **Rules** and **Rule Monitoring** tabl ## Manage alert tag options The `securitySolution:alertTags` field determines which options display in the alert tag menu. The default alert tag options are `Duplicate`, `False Positive`, and `Further investigation required`. You can update the alert tag menu by editing these options or adding more. To learn more about using alert tags, refer to Apply and filter alert tags. + +
+ +## Exclude cold and frozen data tiers from rule executions + +Set the `excludedDataTiersForRuleExecution` field to ensure that events from cold and frozen [data tiers](((ref))/data-tiers.html) aren't searched during rule executions. Excluding data from cold and frozen tiers might help to improve rule performance or reduce execution time. + +If you specify multiple data tiers, values must be separated by commas, for example: `data_frozen`, `data_cold`. This setting is turned off by default. \ No newline at end of file From d42f551931ffc848d3ca5ff060cb196d07ce97e4 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Sun, 22 Sep 2024 18:24:47 -0400 Subject: [PATCH 2/9] Adding docs for individual rules --- docs/detections/rules-ui-create.asciidoc | 6 ++- .../getting-started/advanced-setting.asciidoc | 51 +++++++++++++++++-- .../serverless/settings/advanced-settings.mdx | 10 +--- 3 files changed, 52 insertions(+), 15 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 8b44ed230e..379ca57e5f 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -24,7 +24,11 @@ To create a new detection rule, follow these steps: * You'll also need permissions to enable and view detections, manage rules, manage alerts, and preview rules. These permissions depend on the user role. Refer to <> for more information. -- -TIP: At any step, you can <> before saving it to see what kind of results you can expect. +[TIP] +============== +* At any step, you can <> before saving it to see what kind of results you can expect. +* To ensure rules don't search cold and frozen data when executing, configure the `excludedDataTiersForRuleExecution` advanced setting, or add a {ref}/sql-rest-filtering.html[Query DSL] filter to individual rules. Refer to <> to learn more. +============== NOTE: Additional configuration is required for detection rules using cross-cluster search. Refer to <>. diff --git a/docs/getting-started/advanced-setting.asciidoc b/docs/getting-started/advanced-setting.asciidoc index 9911a0c8cf..7f491f0599 100644 --- a/docs/getting-started/advanced-setting.asciidoc +++ b/docs/getting-started/advanced-setting.asciidoc @@ -179,8 +179,49 @@ The `securitySolution:alertTags` field determines which options display in the a [discrete] [[exclude-cold-frozen-data-rule-executions]] -== Exclude cold and frozen data tiers from rule executions - -Set the `excludedDataTiersForRuleExecution` field to ensure that events from cold and frozen {ref}/data-tiers.html[data tiers] aren't searched during rule executions. Excluding data from cold and frozen tiers might help to improve rule performance or reduce execution time. - -If you specify multiple data tiers, values must be separated by commas, for example: `data_frozen`, `data_cold`. This setting is turned off by default. \ No newline at end of file +== Exclude cold and frozen data from rule executions + +To ensure rules don't search cold and frozen data when executing, specify cold and frozen {ref}/data-tiers.html[data tiers] in the `excludedDataTiersForRuleExecution` field. Multiple data tiers must be separated by commas, for example: `data_frozen`, `data_cold`. This setting is turned off by default; turning it on can improve rule performance and reduce execution time. + +The `excludedDataTiersForRuleExecution` setting does not apply to {esql} or {ml} rules. + +[TIP] +==== + +The `excludedDataTiersForRuleExecution` setting applies to all rules in a {kib} space. To only exclude cold and frozen data from specific rules, add a {ref}/sql-rest-filtering.html[Query DSL] filter to the rules you want affected. Add the filter when creating a new rule or updating an existing one. + +Here is a sample Query DSL filter that excludes frozen tier data from a rule's execution: + +[source,console] +---- +{ + "bool":{ + "must_not":{ + "terms":{ + "_tier":[ + "data_frozen" + ] + } + } + } +} +---- + +Here is another sample Query DSL filter that excludes cold and frozen tier data from a rule's execution: + +[source,console] +---- +{ + "bool":{ + "must_not":{ + "terms":{ + "_tier":[ + "data_frozen", "data_cold" + ] + } + } + } +} +---- + +==== \ No newline at end of file diff --git a/docs/serverless/settings/advanced-settings.mdx b/docs/serverless/settings/advanced-settings.mdx index f8b8cd7117..8120bf6f5f 100644 --- a/docs/serverless/settings/advanced-settings.mdx +++ b/docs/serverless/settings/advanced-settings.mdx @@ -195,12 +195,4 @@ By default, Elastic prebuilt rules in the **Rules** and **Rule Monitoring** tabl ## Manage alert tag options -The `securitySolution:alertTags` field determines which options display in the alert tag menu. The default alert tag options are `Duplicate`, `False Positive`, and `Further investigation required`. You can update the alert tag menu by editing these options or adding more. To learn more about using alert tags, refer to Apply and filter alert tags. - -
- -## Exclude cold and frozen data tiers from rule executions - -Set the `excludedDataTiersForRuleExecution` field to ensure that events from cold and frozen [data tiers](((ref))/data-tiers.html) aren't searched during rule executions. Excluding data from cold and frozen tiers might help to improve rule performance or reduce execution time. - -If you specify multiple data tiers, values must be separated by commas, for example: `data_frozen`, `data_cold`. This setting is turned off by default. \ No newline at end of file +The `securitySolution:alertTags` field determines which options display in the alert tag menu. The default alert tag options are `Duplicate`, `False Positive`, and `Further investigation required`. You can update the alert tag menu by editing these options or adding more. To learn more about using alert tags, refer to Apply and filter alert tags. \ No newline at end of file From 94437e03e5e5134aa7de2b2b237aa58a572f8ad1 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Sun, 22 Sep 2024 18:34:42 -0400 Subject: [PATCH 3/9] Reverting change --- docs/serverless/settings/advanced-settings.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/serverless/settings/advanced-settings.mdx b/docs/serverless/settings/advanced-settings.mdx index 8120bf6f5f..67046cb80a 100644 --- a/docs/serverless/settings/advanced-settings.mdx +++ b/docs/serverless/settings/advanced-settings.mdx @@ -195,4 +195,4 @@ By default, Elastic prebuilt rules in the **Rules** and **Rule Monitoring** tabl ## Manage alert tag options -The `securitySolution:alertTags` field determines which options display in the alert tag menu. The default alert tag options are `Duplicate`, `False Positive`, and `Further investigation required`. You can update the alert tag menu by editing these options or adding more. To learn more about using alert tags, refer to Apply and filter alert tags. \ No newline at end of file +The `securitySolution:alertTags` field determines which options display in the alert tag menu. The default alert tag options are `Duplicate`, `False Positive`, and `Further investigation required`. You can update the alert tag menu by editing these options or adding more. To learn more about using alert tags, refer to Apply and filter alert tags. From 1b68bcc21a0096a0a5061a8f4d29ba6b2d77c0e1 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Mon, 23 Sep 2024 16:57:44 -0400 Subject: [PATCH 4/9] Update docs/getting-started/advanced-setting.asciidoc Co-authored-by: Joe Peeples --- docs/getting-started/advanced-setting.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started/advanced-setting.asciidoc b/docs/getting-started/advanced-setting.asciidoc index 92b7657d83..4faf80b9ae 100644 --- a/docs/getting-started/advanced-setting.asciidoc +++ b/docs/getting-started/advanced-setting.asciidoc @@ -182,12 +182,12 @@ The `securitySolution:alertTags` field determines which options display in the a To ensure rules don't search cold and frozen data when executing, specify cold and frozen {ref}/data-tiers.html[data tiers] in the `excludedDataTiersForRuleExecution` field. Multiple data tiers must be separated by commas, for example: `data_frozen`, `data_cold`. This setting is turned off by default; turning it on can improve rule performance and reduce execution time. -The `excludedDataTiersForRuleExecution` setting does not apply to {esql} or {ml} rules. +This setting does not apply to {esql} or {ml} rules. [TIP] ==== -The `excludedDataTiersForRuleExecution` setting applies to all rules in a {kib} space. To only exclude cold and frozen data from specific rules, add a {ref}/sql-rest-filtering.html[Query DSL] filter to the rules you want affected. Add the filter when creating a new rule or updating an existing one. +This setting applies to all rules in a {kib} space. To only exclude cold and frozen data from specific rules, add a Query DSL filter to the rules you want affected. Add the filter when creating a new rule or updating an existing one. Here is a sample Query DSL filter that excludes frozen tier data from a rule's execution: From 4a6659780a395f7fc6dd7e3400316bb7616ba1ba Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 24 Sep 2024 14:56:15 -0400 Subject: [PATCH 5/9] Adding new topic --- ...ns-exclude-cold-frozen-data-tiers.asciidoc | 49 +++++++++++++++++++ docs/detections/detections-index.asciidoc | 2 + docs/detections/rules-ui-create.asciidoc | 2 +- .../getting-started/advanced-setting.asciidoc | 36 +------------- 4 files changed, 53 insertions(+), 36 deletions(-) create mode 100644 docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc diff --git a/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc new file mode 100644 index 0000000000..a7364c4a97 --- /dev/null +++ b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc @@ -0,0 +1,49 @@ +[[exclude-cold-frozen-data-individual-rules]] += Exclude cold and frozen data from a rule + +:frontmatter-description: Configure a rule to ignore cold and frozen data during execution. +:frontmatter-tags-products: [security] +:frontmatter-tags-content-type: [how-to] +:frontmatter-tags-user-goals: [manage] + +Rules that query cold and frozen data might perform slower and have longer execution times. To exclude cold and frozen data, add a Query DSL filter that forces the rule to ignore cold and frozen {ref}/data-tiers.html[data tiers] when executing. You can add the filter when creating a new rule or updating an existing one. + +NOTE: This method is not supported for {esql} and {ml} rules. + +TIP: To ensure that _all_ rules in a {kib} space exclude cold and frozen data when executing, configure the `excludedDataTiersForRuleExecution` <>. + + +Here is a sample Query DSL filter that excludes frozen tier data from a rule's execution: + +[source,console] +---- +{ + "bool":{ + "must_not":{ + "terms":{ + "_tier":[ + "data_frozen" + ] + } + } + } +} +---- + +Here is another sample Query DSL filter that excludes cold and frozen tier data from a rule's execution: + +[source,console] +---- +{ + "bool":{ + "must_not":{ + "terms":{ + "_tier":[ + "data_frozen", "data_cold" + ] + } + } + } +} +---- + diff --git a/docs/detections/detections-index.asciidoc b/docs/detections/detections-index.asciidoc index e500284b33..2d1cab74d6 100644 --- a/docs/detections/detections-index.asciidoc +++ b/docs/detections/detections-index.asciidoc @@ -11,6 +11,8 @@ include::rules-cross-cluster-search.asciidoc[leveloffset=+1] include::investigation-guide-actions.asciidoc[leveloffset=+1] +include::detections-exclude-cold-frozen-data-tiers.asciidoc[leveloffset=+1] + include::prebuilt-rules-management.asciidoc[] include::rules-ui-manage.asciidoc[] diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 379ca57e5f..c03492ae01 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -27,7 +27,7 @@ To create a new detection rule, follow these steps: [TIP] ============== * At any step, you can <> before saving it to see what kind of results you can expect. -* To ensure rules don't search cold and frozen data when executing, configure the `excludedDataTiersForRuleExecution` advanced setting, or add a {ref}/sql-rest-filtering.html[Query DSL] filter to individual rules. Refer to <> to learn more. +* To ensure rules don't search cold and frozen data when executing, either configure the <> advanced setting (which applies to all rules in a space), or add a > to individual rules. ============== NOTE: Additional configuration is required for detection rules using cross-cluster search. Refer to <>. diff --git a/docs/getting-started/advanced-setting.asciidoc b/docs/getting-started/advanced-setting.asciidoc index 4faf80b9ae..e70b009ced 100644 --- a/docs/getting-started/advanced-setting.asciidoc +++ b/docs/getting-started/advanced-setting.asciidoc @@ -187,40 +187,6 @@ This setting does not apply to {esql} or {ml} rules. [TIP] ==== -This setting applies to all rules in a {kib} space. To only exclude cold and frozen data from specific rules, add a Query DSL filter to the rules you want affected. Add the filter when creating a new rule or updating an existing one. - -Here is a sample Query DSL filter that excludes frozen tier data from a rule's execution: - -[source,console] ----- -{ - "bool":{ - "must_not":{ - "terms":{ - "_tier":[ - "data_frozen" - ] - } - } - } -} ----- - -Here is another sample Query DSL filter that excludes cold and frozen tier data from a rule's execution: - -[source,console] ----- -{ - "bool":{ - "must_not":{ - "terms":{ - "_tier":[ - "data_frozen", "data_cold" - ] - } - } - } -} ----- +This setting applies to all rules in a {kib} space. To only exclude cold and frozen data from specific rules, add a Query DSL filter to the rules you want affected. Refer to <>. ==== \ No newline at end of file From 9cf9abf2c18eb286a136c575dd8a6917c6255628 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 24 Sep 2024 15:15:14 -0400 Subject: [PATCH 6/9] give me an s --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index c03492ae01..33e01285ba 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -27,7 +27,7 @@ To create a new detection rule, follow these steps: [TIP] ============== * At any step, you can <> before saving it to see what kind of results you can expect. -* To ensure rules don't search cold and frozen data when executing, either configure the <> advanced setting (which applies to all rules in a space), or add a > to individual rules. +* To ensure rules don't search cold and frozen data when executing, either configure the <> advanced setting (which applies to all rules in a space), or add a > to individual rules. ============== NOTE: Additional configuration is required for detection rules using cross-cluster search. Refer to <>. From bdf777fead402c0bcb6f63765276c420616cbc29 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 24 Sep 2024 15:45:35 -0400 Subject: [PATCH 7/9] Fixes ref and toc order --- .../detections-exclude-cold-frozen-data-tiers.asciidoc | 4 ++-- docs/detections/rules-ui-create.asciidoc | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc index a7364c4a97..03cac78811 100644 --- a/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc +++ b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc @@ -1,5 +1,5 @@ [[exclude-cold-frozen-data-individual-rules]] -= Exclude cold and frozen data from a rule +== Exclude cold and frozen data from a rule :frontmatter-description: Configure a rule to ignore cold and frozen data during execution. :frontmatter-tags-products: [security] @@ -10,7 +10,7 @@ Rules that query cold and frozen data might perform slower and have longer execu NOTE: This method is not supported for {esql} and {ml} rules. -TIP: To ensure that _all_ rules in a {kib} space exclude cold and frozen data when executing, configure the `excludedDataTiersForRuleExecution` <>. +TIP: To ensure that _all_ rules in a {kib} space exclude cold and frozen data when executing, configure the <<`excludedDataTiersForRuleExecution`,exclude-cold-frozen-data-rule-executions>> advanced setting. Here is a sample Query DSL filter that excludes frozen tier data from a rule's execution: diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 33e01285ba..a7336efe9f 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -27,7 +27,7 @@ To create a new detection rule, follow these steps: [TIP] ============== * At any step, you can <> before saving it to see what kind of results you can expect. -* To ensure rules don't search cold and frozen data when executing, either configure the <> advanced setting (which applies to all rules in a space), or add a > to individual rules. +* To ensure rules don't search cold and frozen data when executing, either configure the <> advanced setting (which applies to all rules in a space), or add a <> to individual rules. ============== NOTE: Additional configuration is required for detection rules using cross-cluster search. Refer to <>. From c2bfbec0becfbb5182188e878dcedf11a3aea25d Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 24 Sep 2024 21:47:34 -0400 Subject: [PATCH 8/9] Fixes ref issues --- .../detections-exclude-cold-frozen-data-tiers.asciidoc | 3 +-- docs/detections/rules-ui-create.asciidoc | 2 +- docs/getting-started/advanced-setting.asciidoc | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc index 03cac78811..49af0fdd73 100644 --- a/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc +++ b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc @@ -10,8 +10,7 @@ Rules that query cold and frozen data might perform slower and have longer execu NOTE: This method is not supported for {esql} and {ml} rules. -TIP: To ensure that _all_ rules in a {kib} space exclude cold and frozen data when executing, configure the <<`excludedDataTiersForRuleExecution`,exclude-cold-frozen-data-rule-executions>> advanced setting. - +TIP: To ensure that _all_ rules in a {kib} space exclude cold and frozen data when executing, configure the `excludedDataTiersForRuleExecution` <>. Here is a sample Query DSL filter that excludes frozen tier data from a rule's execution: diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index a7336efe9f..1c1f404baf 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -27,7 +27,7 @@ To create a new detection rule, follow these steps: [TIP] ============== * At any step, you can <> before saving it to see what kind of results you can expect. -* To ensure rules don't search cold and frozen data when executing, either configure the <> advanced setting (which applies to all rules in a space), or add a <> to individual rules. +* To ensure rules don't search cold and frozen data when executing, either configure the `excludedDataTiersForRuleExecution` <> (which applies to all rules in a space), or add a <> to individual rules. ============== NOTE: Additional configuration is required for detection rules using cross-cluster search. Refer to <>. diff --git a/docs/getting-started/advanced-setting.asciidoc b/docs/getting-started/advanced-setting.asciidoc index e70b009ced..02b838b604 100644 --- a/docs/getting-started/advanced-setting.asciidoc +++ b/docs/getting-started/advanced-setting.asciidoc @@ -187,6 +187,6 @@ This setting does not apply to {esql} or {ml} rules. [TIP] ==== -This setting applies to all rules in a {kib} space. To only exclude cold and frozen data from specific rules, add a Query DSL filter to the rules you want affected. Refer to <>. +This setting applies to all rules in a {kib} space. To only exclude cold and frozen data from specific rules, add a <> to the rules you want affected. ==== \ No newline at end of file From 31a2c36a98df492c3a59afcba229f5cd87ac914b Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Wed, 25 Sep 2024 11:00:41 -0400 Subject: [PATCH 9/9] Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc Co-authored-by: Joe Peeples --- .../detections-exclude-cold-frozen-data-tiers.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc index 49af0fdd73..684f51d5a6 100644 --- a/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc +++ b/docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc @@ -6,7 +6,7 @@ :frontmatter-tags-content-type: [how-to] :frontmatter-tags-user-goals: [manage] -Rules that query cold and frozen data might perform slower and have longer execution times. To exclude cold and frozen data, add a Query DSL filter that forces the rule to ignore cold and frozen {ref}/data-tiers.html[data tiers] when executing. You can add the filter when creating a new rule or updating an existing one. +Rules that query cold and frozen data might perform more slowly. To exclude cold and frozen data, add a Query DSL filter that ignores cold and frozen {ref}/data-tiers.html[data tiers] when executing. You can add the filter when creating a new rule or updating an existing one. NOTE: This method is not supported for {esql} and {ml} rules.