From bf1f69cc13d475c968bfaa0c074ce343008bb829 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> Date: Wed, 8 Jan 2025 15:00:01 +0000 Subject: [PATCH 1/2] New endpoint rules (#6100) * New endpoint rules * Adds new page to serverless * updates references in serverless * links to new page * more updates * Address feedback * Address feedback * Adds links * Uncomment endpoint rules in release highlights * Address feedback * Address feedback * Address feedback (cherry picked from commit 066441b4680ccbcc8bd97f7d35bea83e14876509) # Conflicts: # docs/serverless/index.asciidoc # docs/serverless/rules/add-exceptions.asciidoc # docs/serverless/rules/detection-engine-overview.asciidoc # docs/serverless/rules/rules-ui-create.asciidoc --- docs/detections/add-exceptions.asciidoc | 20 +- .../detection-engine-intro.asciidoc | 16 +- docs/detections/rules-ui-create.asciidoc | 8 +- .../admin/endpoint-protection-rules.asciidoc | 43 + docs/management/manage-intro.asciidoc | 1 + .../endpoint-protection-rules.asciidoc | 43 + docs/serverless/index.asciidoc | 201 ++++ docs/serverless/rules/add-exceptions.asciidoc | 285 ++++++ .../rules/detection-engine-overview.asciidoc | 134 +++ .../serverless/rules/rules-ui-create.asciidoc | 896 ++++++++++++++++++ 10 files changed, 1615 insertions(+), 32 deletions(-) create mode 100644 docs/management/admin/endpoint-protection-rules.asciidoc create mode 100644 docs/serverless/edr-manage/endpoint-protection-rules.asciidoc create mode 100644 docs/serverless/index.asciidoc create mode 100644 docs/serverless/rules/add-exceptions.asciidoc create mode 100644 docs/serverless/rules/detection-engine-overview.asciidoc create mode 100644 docs/serverless/rules/rules-ui-create.asciidoc diff --git a/docs/detections/add-exceptions.asciidoc b/docs/detections/add-exceptions.asciidoc index 80cc93875f..55d1eece63 100644 --- a/docs/detections/add-exceptions.asciidoc +++ b/docs/detections/add-exceptions.asciidoc @@ -129,22 +129,16 @@ Closes all alerts that match the exception's conditions and were generated only [[endpoint-rule-exceptions]] === Add {elastic-endpoint} exceptions -Like detection rule exceptions, you can add Endpoint agent exceptions either by editing the Endpoint Security rule or by adding them as actions on alerts generated by the Endpoint Security rule. {elastic-endpoint} alerts have the following fields: +You can add {elastic-endpoint} exceptions to <> or to rules that are associated with {elastic-endpoint} rule exceptions. To associate rules when creating or editing a rule, select the <> option. -* `kibana.alert.original_event.module determined:endpoint` -* `kibana.alert.original_event.kind:alert` - -You can also add Endpoint exceptions to rules that are associated with {elastic-endpoint} rule exceptions. To associate rules when creating or editing a rule, select the <> option. - -Endpoint exceptions are added to the Endpoint Security rule *and* the {elastic-endpoint} on your hosts. +Endpoint exceptions are added to the endpoint protection rules *and* the {elastic-endpoint} on your hosts. [IMPORTANT] ============= -Exceptions added to the Endpoint Security rule affect all alerts sent -from the Endpoint agent. Be careful not to unintentionally prevent useful Endpoint -alerts. +Exceptions added to the endpoint protection rules affect all alerts sent +from {elastic-endpoint}. Be careful not to unintentionally prevent useful Endpoint alerts. -Additionally, to add an Endpoint exception to the Endpoint Security rule, there must be at least one Endpoint Security alert generated in the system. For non-production use, if no alerts exist, you can trigger a test alert using malware emulation techniques or tools such as the Anti Malware Testfile from the https://www.eicar.org/[European Institute for Computer Anti-Virus Research (EICAR)]. +Additionally, to add an Endpoint exception to an endpoint protection rule, there must be at least one {elastic-endpoint} alert generated in the system. For non-production use, if no alerts exist, you can trigger a test alert using malware emulation techniques or tools such as the Anti Malware Testfile from the https://www.eicar.org/[European Institute for Computer Anti-Virus Research (EICAR)]. ============= [IMPORTANT] @@ -158,7 +152,7 @@ Additionally, to add an Endpoint exception to the Endpoint Security rule, there * To add an Endpoint exception from the rule details page: .. Find *Detection rules (SIEM)* in the navigation menu or by using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. -.. In the Rules table, search for and select the Elastic *Endpoint Security* rule. +.. In the Rules table, search for and select one of the <>. .. Scroll down the rule details page, select the *Endpoint exceptions* tab, then click *Add endpoint exception*. * To add an Endpoint exception from the Alerts table: @@ -170,7 +164,7 @@ alert, click the *More actions* menu (*...*), then select *Add Endpoint exceptio .. Find the *Shared exception lists* page in the navigation menu or by using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. .. Expand the Endpoint Security Exception List or click the list name to open the list's details page. Next, click *Add endpoint exception*. + -NOTE: The Endpoint Security Exception List is automatically created. By default, it's associated with the Endpoint Security rule and any rules with the <> option selected. +NOTE: The Endpoint Security Exception List is automatically created. By default, it's associated with endpoint protection rules and any rules with the <> option selected. -- + diff --git a/docs/detections/detection-engine-intro.asciidoc b/docs/detections/detection-engine-intro.asciidoc index 105e2ade3e..f649a98a16 100644 --- a/docs/detections/detection-engine-intro.asciidoc +++ b/docs/detections/detection-engine-intro.asciidoc @@ -22,21 +22,9 @@ how to modify the rules to reduce false positives and get a better set of actionable alerts. You can also use exceptions and value lists when creating or modifying your own rules. -There are two special prebuilt rules you need to know about: +There are several special prebuilt rules you need to know about: -* <>: -Automatically creates an alert from all incoming Elastic Endpoint alerts. To -receive Elastic Endpoint alerts, you must install the Endpoint agent on your -hosts (see <>). -+ -When this rule is enabled, the following Endpoint events are displayed as -detection alerts: -+ -** Malware Prevention Alert -** Malware Detection Alert -+ -NOTE: When you load the prebuilt rules, this is the only rule that is enabled -by default. +* <>: Automatically create alerts based on {elastic-defend}'s threat monitoring and prevention. * <>: Automatically creates an alert for all incoming third-party system alerts (for example, Suricata alerts). diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 1638a6664c..cf20dd03be 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -563,13 +563,11 @@ After you create the rule, you can find all custom highlighted fields in the Abo alerts created by the rule. You can also add action buttons to <> or <> using alert data. .. *Author* (optional): The rule's authors. .. *License* (optional): The rule's license. -.. *Elastic endpoint exceptions* (optional): Adds all Elastic Endpoint Security -rule exceptions to this rule (refer to <> to learn more about adding endpoint exceptions). +.. *Elastic endpoint exceptions* (optional): Adds all <> to this rule. + NOTE: If you select this option, you can add -<> on the Rule details page. -Additionally, all future exceptions added to the Endpoint Security rule -also affect this rule. +{elastic-endpoint} exceptions on the Rule details page. +Additionally, all future exceptions added to <> will also affect this rule. + .. *Building block* (optional): Select to create a building-block rule. By diff --git a/docs/management/admin/endpoint-protection-rules.asciidoc b/docs/management/admin/endpoint-protection-rules.asciidoc new file mode 100644 index 0000000000..c79c57e663 --- /dev/null +++ b/docs/management/admin/endpoint-protection-rules.asciidoc @@ -0,0 +1,43 @@ +[[endpoint-protection-rules]] += Endpoint protection rules + +Endpoint protection rules are <> designed to help you manage and respond to alerts generated by {elastic-endpoint}, the installed component that performs {elastic-defend}'s threat monitoring and prevention. These rules include the <> rule as well as additional detection and prevention rules for different {elastic-defend} protection features. + +IMPORTANT: To receive {elastic-endpoint} alerts, you must install {agent} and the {elastic-defend} integration on your hosts (refer to <>). + +When endpoint protection rules are triggered, {elastic-endpoint} alerts are displayed as detection alerts in the {security-app}. The detection alert name is taken from the {elastic-endpoint} alert message and overwrites the prebuilt rule name in the Alerts table. For example, for malware protection, the following {elastic-endpoint} alerts are displayed as detection alerts: + +** Malware Prevention Alert +** Malware Detection Alert + +[discrete] +[[endpoint-sec-rule]] +== Endpoint Security rule + +The Endpoint Security rule automatically creates an alert from all incoming {elastic-endpoint} alerts. + +NOTE: When you install Elastic prebuilt rules, the {elastic-defend} is enabled by default. + +[discrete] +[[feature-protection-rules]] +== Feature-specific protection rules + +The following endpoint protection rules give you more granular control over how you handle the generated alerts. These rules are tailored for each of {elastic-defend}'s endpoint protection features—malware, ransomware, memory threats, and malicious behavior. Enabling these rules allows you to configure more specific actions based on the protection feature and whether the malicious activity was prevented or detected. + +* Behavior - Detected - Elastic Defend +* Behavior - Prevented - Endpoint Defend +* Malicious File - Detected - Elastic Defend +* Malicious File - Prevented - Elastic Defend +* Memory Signature - Detected - Elastic Defend +* Memory Signature - Prevented - Elastic Defend +* Ransomware - Detected - Elastic Defend +* Ransomware - Prevented - Elastic Defend + +NOTE: If you choose to use the feature-specific protection rules, we recommend that you disable the Endpoint Security rule, as using both will result in duplicate alerts. + +To use these rules, you need to manually enable them from the **Rules** page in the {security-app}. Follow the instructions for <>. + +[discrete] +== Endpoint security exception handling + +All endpoint protection rules share a common exception list called the Endpoint Security Exception List. This ensures that if you switch between using the Endpoint Security rule and the feature-specific protection rules, your existing <> continue to apply. \ No newline at end of file diff --git a/docs/management/manage-intro.asciidoc b/docs/management/manage-intro.asciidoc index 45e8abcb1c..e284a64348 100644 --- a/docs/management/manage-intro.asciidoc +++ b/docs/management/manage-intro.asciidoc @@ -12,6 +12,7 @@ include::{security-docs-root}/docs/management/admin/host-isolation-exceptions.as include::{security-docs-root}/docs/management/admin/blocklist.asciidoc[leveloffset=+1] include::{security-docs-root}/docs/management/admin/endpoint-artifacts.asciidoc[leveloffset=+1] include::{security-docs-root}/docs/management/admin/endpoint-event-capture.asciidoc[leveloffset=+1] +include::{security-docs-root}/docs/management/admin/endpoint-protection-rules.asciidoc[leveloffset=+1] include::{security-docs-root}/docs/management/admin/allowlist-endpoint-3rd-party-av.asciidoc[leveloffset=+1] include::{security-docs-root}/docs/management/admin/endpoint-self-protection.asciidoc[leveloffset=+1] include::{security-docs-root}/docs/management/admin/endpoint-command-ref.asciidoc[leveloffset=+1] diff --git a/docs/serverless/edr-manage/endpoint-protection-rules.asciidoc b/docs/serverless/edr-manage/endpoint-protection-rules.asciidoc new file mode 100644 index 0000000000..f71d61c72f --- /dev/null +++ b/docs/serverless/edr-manage/endpoint-protection-rules.asciidoc @@ -0,0 +1,43 @@ +[[endpoint-protection-rules]] += Endpoint protection rules + +Endpoint protection rules are <> designed to help you manage and respond to alerts generated by {elastic-endpoint}, the installed component that performs {elastic-defend}'s threat monitoring and prevention. These rules include the Endpoint Security rule as well as additional detection and prevention rules for different {elastic-defend} protection features. + +IMPORTANT: To receive {elastic-endpoint} alerts, you must install {agent} and the {elastic-defend} integration on your hosts (refer to <>). + +When endpoint protection rules are triggered, {elastic-endpoint} alerts are displayed as detection alerts in the {security-app}. The detection alert name is taken from the {elastic-endpoint} alert message and overwrites the prebuilt rule name in the Alerts table. For example, for malware protection, the following {elastic-endpoint} alerts are displayed as detection alerts: + +** Malware Prevention Alert +** Malware Detection Alert + +[discrete] +[[endpoint-sec-rule]] +== Endpoint Security rule + +The Endpoint Security rule automatically creates an alert from all incoming {elastic-endpoint} alerts. + +NOTE: When you install Elastic prebuilt rules, the Endpoint Security rule that is enabled by default. + +[discrete] +[[feature-protection-rules]] +== Feature-specific protection rules + +The following endpoint protection rules give you more granular control over how you handle the generated alerts. These rules are tailored for each of {elastic-defend}'s endpoint protection features—malware, ransomware, memory threats, and malicious behavior. Enabling these rules allows you to configure more specific actions based on the protection feature and whether the malicious activity was prevented or detected. + +* Behavior - Detected - Elastic Defend +* Behavior - Prevented - Endpoint Defend +* Malicious File - Detected - Elastic Defend +* Malicious File - Prevented - Elastic Defend +* Memory Signature - Detected - Elastic Defend +* Memory Signature - Prevented - Elastic Defend +* Ransomware - Detected - Elastic Defend +* Ransomware - Prevented - Elastic Defend + +NOTE: If you choose to use the feature-specific protection rules, we recommend that you disable the Endpoint Security rule, as using both will result in duplicate alerts. + +To use these rules, you need to manually enable them from the **Rules** page in the {security-app}. Follow the instructions for <>. + +[discrete] +== Endpoint security exception handling + +All endpoint protection rules share a common exception list called the Endpoint Security Exception List. This ensures that if you switch between using the Endpoint Security rule and the feature-specific protection rules, your existing <> continue to apply. \ No newline at end of file diff --git a/docs/serverless/index.asciidoc b/docs/serverless/index.asciidoc new file mode 100644 index 0000000000..83e32bbd4e --- /dev/null +++ b/docs/serverless/index.asciidoc @@ -0,0 +1,201 @@ +:doctype: book + +include::{asciidoc-dir}/../../shared/versions/stack/master.asciidoc[] +include::{asciidoc-dir}/../../shared/attributes.asciidoc[] + +[[what-is-security-serverless]] +== {sec-serverless} + +++++ +Elastic Security +++++ + +include::./what-is-security-serverless.asciidoc[leveloffset=+2] + +include::./security-overview.asciidoc[leveloffset=+2] + +include::./billing.asciidoc[leveloffset=+2] + +include::./projects-create/create-project.asciidoc[leveloffset=+2] + +include::./sec-requirements.asciidoc[leveloffset=+2] + +include::./security-ui.asciidoc[leveloffset=+2] +include::./security-spaces.asciidoc[leveloffset=+3] + +include::./AI-for-security/ai-for-security-landing-pg.asciidoc[leveloffset=+2] +include::./AI-for-security/ai-assistant.asciidoc[leveloffset=+3] +include::./AI-for-security/knowledge-base.asciidoc[leveloffset=+4] +include::./AI-for-security/attack-discovery.asciidoc[leveloffset=+3] +include::./AI-for-security/llm-connector-guides.asciidoc[leveloffset=+3] +include::./AI-for-security/llm-performance-matrix.asciidoc[leveloffset=+4] +include::./AI-for-security/connect-to-azure-openai.asciidoc[leveloffset=+4] +include::./AI-for-security/connect-to-bedrock.asciidoc[leveloffset=+4] +include::./AI-for-security/connect-to-openai.asciidoc[leveloffset=+4] +include::./AI-for-security/connect-to-vertex.asciidoc[leveloffset=+4] +include::./AI-for-security/connect-to-byo-llm.asciidoc[leveloffset=+4] +include::./AI-for-security/ai-use-cases.asciidoc[leveloffset=+3] +include::./AI-for-security/usecase-attack-disc-ai-assistant-incident-reporting.asciidoc[leveloffset=+4] +include::./AI-for-security/ai-assistant-alert-triage.asciidoc[leveloffset=+4] +include::./AI-for-security/ai-assistant-esql-queries.asciidoc[leveloffset=+4] + +include::./ingest/ingest-data.asciidoc[leveloffset=+2] +include::./ingest/threat-intelligence.asciidoc[leveloffset=+3] +include::./ingest/auto-import.asciidoc[leveloffset=+3] +include::./ingest/agentless-integrations.asciidoc[leveloffset=+3] +include::./ingest/agentless-troubleshooting.asciidoc[leveloffset=+4] + +include::./edr-install-config/endpoint-protection-intro.asciidoc[leveloffset=+2] +include::./edr-install-config/deploy-endpoint-reqs.asciidoc[leveloffset=+3] +include::./edr-install-config/install-elastic-defend.asciidoc[leveloffset=+3] +include::./edr-install-config/deploy-endpoint-macos-cat-mont.asciidoc[leveloffset=+4] +include::./edr-install-config/deploy-endpoint-macos-ven.asciidoc[leveloffset=+4] +include::./edr-install-config/deploy-with-mdm.asciidoc[leveloffset=+4] +include::./edr-install-config/agent-tamper-protection.asciidoc[leveloffset=+4] +include::./edr-install-config/defend-feature-privs.asciidoc[leveloffset=+3] +include::./edr-install-config/configure-endpoint-integration-policy.asciidoc[leveloffset=+3] +include::./edr-install-config/artifact-control.asciidoc[leveloffset=+4] +include::./edr-install-config/endpoint-diagnostic-data.asciidoc[leveloffset=+4] +include::./edr-install-config/self-healing-rollback.asciidoc[leveloffset=+4] +include::./edr-install-config/linux-file-monitoring.asciidoc[leveloffset=+4] +include::./edr-install-config/endpoint-data-volume.asciidoc[leveloffset=+4] +include::./edr-install-config/uninstall-agent.asciidoc[leveloffset=+3] + +include::./edr-manage/manage-endpoint-protection.asciidoc[leveloffset=+2] +include::./edr-manage/endpoints-page.asciidoc[leveloffset=+3] +include::./edr-manage/policies-page-ov.asciidoc[leveloffset=+3] +include::./edr-manage/trusted-apps-ov.asciidoc[leveloffset=+3] +include::./edr-manage/event-filters.asciidoc[leveloffset=+3] +include::./edr-manage/host-isolation-exceptions.asciidoc[leveloffset=+3] +include::./edr-manage/blocklist.asciidoc[leveloffset=+3] +include::./edr-manage/optimize-edr.asciidoc[leveloffset=+3] +include::./edr-manage/endpoint-event-capture.asciidoc[leveloffset=+3] +include::./edr-manage/endpoint-protection-rules.asciidoc[leveloffset=+3] +include::./edr-manage/allowlist-endpoint-3rd-party-av.asciidoc[leveloffset=+3] +include::./edr-manage/endpoint-self-protection.asciidoc[leveloffset=+3] +include::./edr-manage/endpoint-command-ref.asciidoc[leveloffset=+3] + +include::./endpoint-response-actions/response-actions.asciidoc[leveloffset=+2] +include::./endpoint-response-actions/automated-response-actions.asciidoc[leveloffset=+3] +include::./endpoint-response-actions/host-isolation-ov.asciidoc[leveloffset=+3] +include::./endpoint-response-actions/response-actions-history.asciidoc[leveloffset=+3] +include::./endpoint-response-actions/third-party-actions.asciidoc[leveloffset=+3] +include::./endpoint-response-actions/response-actions-config.asciidoc[leveloffset=+3] + +include::./cloud-native-security/cloud-native-security-overview.asciidoc[leveloffset=+2] +include::./cloud-native-security/security-posture-management.asciidoc[leveloffset=+3] +include::./cloud-native-security/enable-cloudsec.asciidoc[leveloffset=+3] +include::./cloud-native-security/cspm.asciidoc[leveloffset=+3] +include::./cloud-native-security/cspm-get-started.asciidoc[leveloffset=+4] +include::./cloud-native-security/cspm-get-started-gcp.asciidoc[leveloffset=+4] +include::./cloud-native-security/cspm-get-started-azure.asciidoc[leveloffset=+4] +include::./cloud-native-security/cspm-permissions.asciidoc[leveloffset=+4] +include::./cloud-native-security/cspm-findings-page.asciidoc[leveloffset=+4] +include::./cloud-native-security/benchmark-rules.asciidoc[leveloffset=+4] +include::./cloud-native-security/cspm-cloud-posture-dashboard-dash.asciidoc[leveloffset=+4] +include::./cloud-native-security/cspm-security-posture-faq.asciidoc[leveloffset=+4] +include::./cloud-native-security/kspm.asciidoc[leveloffset=+3] +include::./cloud-native-security/get-started-with-kspm.asciidoc[leveloffset=+4] +include::./cloud-native-security/kspm-cspm-findings-page.asciidoc[leveloffset=+4] +include::./cloud-native-security/kspm-benchmark-rules.asciidoc[leveloffset=+4] +include::./cloud-native-security/kspm-cloud-posture-dashboard-dash.asciidoc[leveloffset=+4] +include::./cloud-native-security/security-posture-faq.asciidoc[leveloffset=+4] +include::./cloud-native-security/vuln-management-overview.asciidoc[leveloffset=+3] +include::./cloud-native-security/vuln-management-get-started.asciidoc[leveloffset=+4] +include::./cloud-native-security/vuln-management-findings.asciidoc[leveloffset=+4] +include::./cloud-native-security/vuln-management-dashboard-dash.asciidoc[leveloffset=+4] +include::./cloud-native-security/vuln-management-faq.asciidoc[leveloffset=+4] +include::./cloud-native-security/cloud-workload-protection.asciidoc[leveloffset=+3] +include::./cloud-native-security/environment-variable-capture.asciidoc[leveloffset=+4] +include::./cloud-native-security/ingest-cncf-data.asciidoc[leveloffset=+3] +include::./cloud-native-security/falco-setup.asciidoc[leveloffset=+4] +include::./cloud-native-security/aws-securityhub.asciidoc[leveloffset=+4] +include::./cloud-native-security/wiz.asciidoc[leveloffset=+4] + +include::./explore/explore-your-data.asciidoc[leveloffset=+2] +include::./explore/hosts-overview.asciidoc[leveloffset=+3] +include::./explore/network-page-overview.asciidoc[leveloffset=+3] +include::./explore/conf-map-ui.asciidoc[leveloffset=+4] +include::./explore/users-page.asciidoc[leveloffset=+3] +include::./explore/data-views-in-sec.asciidoc[leveloffset=+3] +include::./explore/runtime-fields.asciidoc[leveloffset=+3] +include::./explore/siem-field-reference.asciidoc[leveloffset=+3] + +include::./dashboards/dashboards-overview.asciidoc[leveloffset=+2] +include::./dashboards/overview-dashboard.asciidoc[leveloffset=+3] +include::./dashboards/detection-response-dashboard.asciidoc[leveloffset=+3] +include::./dashboards/cloud-posture-dashboard-dash.asciidoc[leveloffset=+3] +include::./dashboards/detection-entity-dashboard.asciidoc[leveloffset=+3] +include::./dashboards/data-quality-dash.asciidoc[leveloffset=+3] +include::./dashboards/vuln-management-dashboard-dash.asciidoc[leveloffset=+3] +include::./dashboards/rule-monitoring-dashboard.asciidoc[leveloffset=+3] + +include::./rules/detection-engine-overview.asciidoc[leveloffset=+2] +include::./rules/detections-permissions-section.asciidoc[leveloffset=+3] +include::./rules/detections-logsdb-impact.asciidoc[leveloffset=+3] + +include::./rules/about-rules.asciidoc[leveloffset=+2] +include::./rules/rules-ui-create.asciidoc[leveloffset=+3] +include::./rules/interactive-investigation-guides.asciidoc[leveloffset=+4] +include::./rules/building-block-rule.asciidoc[leveloffset=+4] +include::./rules/prebuilt-rules/prebuilt-rules-management.asciidoc[leveloffset=+3] +include::./rules/rules-ui-management.asciidoc[leveloffset=+3] +include::./rules/alerts-ui-monitor.asciidoc[leveloffset=+3] +include::./rules/detections-ui-exceptions.asciidoc[leveloffset=+3] +include::./rules/value-lists-exceptions.asciidoc[leveloffset=+4] +include::./rules/add-exceptions.asciidoc[leveloffset=+4] +include::./rules/shared-exception-lists.asciidoc[leveloffset=+4] +include::./rules/rules-coverage.asciidoc[leveloffset=+3] +include::./rules/tuning-detection-signals.asciidoc[leveloffset=+3] +include::./rules/prebuilt-rules/prebuilt-rules.asciidoc[leveloffset=+3] + +include::./alerts/alerts-ui-manage.asciidoc[leveloffset=+2] +include::./alerts/visualize-alerts.asciidoc[leveloffset=+3] +include::./alerts/view-alert-details.asciidoc[leveloffset=+3] +include::./alerts/signals-to-cases.asciidoc[leveloffset=+3] +include::./alerts/alert-suppression.asciidoc[leveloffset=+3] +include::./alerts/reduce-notifications-alerts.asciidoc[leveloffset=+3] +include::./alerts/query-alert-indices.asciidoc[leveloffset=+3] +include::./alerts/alert-schema.asciidoc[leveloffset=+3] + +include::./advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc[leveloffset=+2] +include::./advanced-entity-analytics/entity-risk-scoring.asciidoc[leveloffset=+3] +include::./advanced-entity-analytics/ers-req.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/asset-criticality.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/turn-on-risk-engine.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/analyze-risk-score-data.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/advanced-behavioral-detections.asciidoc[leveloffset=+3] +include::./advanced-entity-analytics/ml-requirements.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/machine-learning.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/tuning-anomaly-results.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/behavioral-detection-use-cases.asciidoc[leveloffset=+4] +include::./advanced-entity-analytics/prebuilt-ml-jobs.asciidoc[leveloffset=+4] + +include::./investigate/investigate-events.asciidoc[leveloffset=+2] +include::./investigate/timelines-ui.asciidoc[leveloffset=+3] +include::./investigate/timeline-templates-ui.asciidoc[leveloffset=+4] +include::./investigate/timeline-object-schema.asciidoc[leveloffset=+4] +include::./alerts/visual-event-analyzer.asciidoc[leveloffset=+3] +include::./cloud-native-security/session-view.asciidoc[leveloffset=+3] +include::./osquery/use-osquery.asciidoc[leveloffset=+3] +include::./osquery/osquery-response-action.asciidoc[leveloffset=+4] +include::./osquery/invest-guide-run-osquery.asciidoc[leveloffset=+4] +include::./osquery/alerts-run-osquery.asciidoc[leveloffset=+4] +include::./osquery/view-osquery-results.asciidoc[leveloffset=+4] +include::./osquery/osquery-placeholder-fields.asciidoc[leveloffset=+4] +include::./investigate/add-manage-notes.asciidoc[leveloffset=+3] +include::./investigate/indicators-of-compromise.asciidoc[leveloffset=+3] +include::./investigate/cases-overview.asciidoc[leveloffset=+3] +include::./investigate/case-permissions.asciidoc[leveloffset=+4] +include::./investigate/cases-open-manage.asciidoc[leveloffset=+4] +include::./investigate/cases-settings.asciidoc[leveloffset=+4] + +include::./assets/asset-management.asciidoc[leveloffset=+2] + +include::./settings/manage-settings.asciidoc[leveloffset=+2] +include::./settings/project-settings.asciidoc[leveloffset=+3] +include::./settings/advanced-settings.asciidoc[leveloffset=+3] + +include::./troubleshooting/troubleshooting-intro.asciidoc[leveloffset=+2] +include::./troubleshooting/ts-detection-rules.asciidoc[leveloffset=+3] +include::./troubleshooting/troubleshoot-endpoints.asciidoc[leveloffset=+3] \ No newline at end of file diff --git a/docs/serverless/rules/add-exceptions.asciidoc b/docs/serverless/rules/add-exceptions.asciidoc new file mode 100644 index 0000000000..5013dc4db8 --- /dev/null +++ b/docs/serverless/rules/add-exceptions.asciidoc @@ -0,0 +1,285 @@ +[[security-add-exceptions]] += Add and manage exceptions + +// :description: Learn how to create and manage rule exceptions. +// :keywords: serverless, security, how-to, configure + + +You can add exceptions to a rule from the rule details page, the Alerts table, the alert details flyout, or the Shared Exception Lists page. When you add an exception, you can also close all alerts that meet the exception’s criteria. + +[IMPORTANT] +==== +* To ensure an exception is successfully applied, ensure that the fields you've defined for its query are correctly and consistently mapped in their respective indices. Refer to {ecs-ref}[ECS] to learn more about supported mappings. +* Be careful when adding exceptions to <> rules. Exceptions are evaluated against every event in the sequence, and if an exception matches any events that are necessary to complete the sequence, alerts are not created. ++ +To exclude values from a +specific event in the sequence, update the rule's EQL statement. For example: ++ +[source,eql] +---- +`sequence + [file where file.extension == "exe" + and file.name != "app-name.exe"] + [process where true + and process.name != "process-name.exe"]` +---- +* Be careful when adding exceptions to <> rules. Exceptions are evaluated against source and indicator indices, so if the exception matches events in _either_ index, alerts are not generated. +==== + +[discrete] +[[detection-rule-exceptions]] +== Add exceptions to a rule + +. Do one of the following: ++ +** To add an exception from the rule details page: ++ +... Go to the rule details page of the rule to which you want to add an +exception (**Rules** → **Detection rules (SIEM)** → **_Rule name_**). +... Scroll down the rule details page, select the **Rule exceptions** tab, then click **Add rule exception**. ++ +[role="screenshot"] +image::images/add-exceptions/-detections-rule-exception-tab.png[Detail of rule exceptions tab] ++ +** To add an exception from the Alerts table: ++ +... Go to **Alerts**. +... Scroll down to the Alerts table, go to the alert you want to create an exception for, click the **More Actions** menu (image:images/icons/boxesHorizontal.svg[Actions menu]), then select **Add rule exception**. +** To add an exception from the alert details flyout: ++ +... Go to **Alerts**. +... Click the **View details** button from the Alerts table. +... In the alert details flyout, click **Take action → Add rule exception**. +** To add an exception from the Shared Exception Lists page: ++ +... Go to **Rules** → **Shared exception lists**. +... Click **Create shared exception list** → **Create exception item**. +. In the **Add rule exception** flyout, name the exception. +. Add conditions that define the exception. When the exception's query evaluates to `true`, rules don't generate alerts even when their criteria are met. ++ +[IMPORTANT] +==== +Rule exceptions are case-sensitive, which means that any character that's entered as an uppercase or lowercase letter will be treated as such. In the event you _don't_ want a field evaluated as case-sensitive, some ECS fields have a `.caseless` version that you can use. +==== ++ +[NOTE] +==== +When you create a new exception from an alert, exception conditions are auto-populated with relevant alert data. Data from custom highlighted fields is listed first. A comment that describes the auto-generated exception conditions is also added to the **Add comments** section. +==== ++ +.. **Field**: Select a field to identify the event being filtered. ++ +[NOTE] +==== +A warning displays for fields with conflicts. Using these fields might cause unexpected exceptions behavior. Refer to <> for more information. +==== +.. **Operator**: Select an operator to define the condition: ++ +*** `is` | `is not` — Must be an exact match of the defined value. +*** `is one of` | `is not one of` — Matches any of the defined values. +*** `exists` | `does not exist` — The field exists. +*** `is in list` | `is not in list` — Matches values in a value list. ++ +[NOTE] +==== +* An exception defined by a value list must use `is in list` or `is not in list` in all conditions. +* Wildcards are not supported in value lists. +* If a value list can't be used due to <>, it'll be unavailable in the **Value** menu. +==== +*** `matches` | `does not match` — Allows you to use wildcards in **Value**, such as `C:\path\*\app.exe`. Available wildcards are `?` (match one character) and `*` (match zero or more characters). The selected **Field** data type must be {ref}/keyword.html#keyword-field-type[keyword], {ref}/text.html#text-field-type[text], or {ref}/keyword.html#wildcard-field-type[wildcard]. ++ +[NOTE] +==== +Some characters must be escaped with a backslash, such as `\` for a literal backslash, `*` for an asterisk, and `\?` for a question mark. Windows paths must be divided with double backslashes (for example, `C:\Windows\explorer.exe`), and paths that already include double backslashes might require four backslashes for each divider. +==== ++ +[IMPORTANT] +==== +Using wildcards can impact performance. To create a more efficient exception using wildcards, use multiple conditions and make them as specific as possible. For example, adding conditions using `process.name` or `file.name` can help limit the scope of wildcard matching. +==== +.. **Value**: Enter the value associated with the **Field**. To enter multiple values (when using `is one of` or `is not one of`), enter each value, then press **Return**. ++ +[NOTE] +==== +Identical, case-sensitive values are supported for the `is one of` and `is not one of` operators. For example, if you want to match the values `Windows` and `windows`, add both values to the **Value** field. +==== ++ +In the following example, the exception was created from the Rules page and prevents the rule from generating alerts when the `svchost.exe` process runs on hostname `siem-kibana`. ++ +[role="screenshot"] +image::images/add-exceptions/-detections-add-exception-ui.png[] +. Click **AND** or **OR** to create multiple conditions and define their relationships. +. Click **Add nested condition** to create conditions using nested fields. This is only required for +<>. For all other fields, nested conditions should not be used. +. Choose to add the exception to a rule or a shared exception list. ++ +[NOTE] +==== +If you are creating an exception from the Shared Exception Lists page, you can add the exception to multiple rules. +==== ++ +[TIP] +==== +If a shared exception list doesn't exist, you can <> from the Shared Exception Lists page. +==== +. (Optional) Enter a comment describing the exception. +. (Optional) Enter a future expiration date and time for the exception. +. Select one of the following alert actions: ++ +** **Close this alert**: Closes the alert when the exception is added. This option +is only available when adding exceptions from the Alerts table. +** **Close all alerts that match this exception and were generated by this rule**: Closes all alerts that match the exception's conditions and were generated only by the current rule. +. Click **Add rule exception**. + +[discrete] +[[endpoint-rule-exceptions]] +== Add {elastic-endpoint} exceptions + +You can add {elastic-endpoint} exceptions to <> or to rules that are associated with {elastic-endpoint} rule exceptions. To associate rules when creating or editing a rule, select the <> option. + +Endpoint exceptions are added to the endpoint protection rules **and** the {elastic-endpoint} on your hosts. + +[IMPORTANT] +==== +Exceptions added to the endpoint protection rules affect all alerts sent +from {elastic-endpoint}. Be careful not to unintentionally prevent useful Endpoint alerts. + +Additionally, to add an Endpoint exception to an endpoint protection rule, there must be at least one {elastic-endpoint} alert generated in the system. For non-production use, if no alerts exist, you can trigger a test alert using malware emulation techniques or tools such as the Anti Malware Testfile from the https://www.eicar.org/[European Institute for Computer Anti-Virus Research (EICAR)]. +==== + +[IMPORTANT] +==== +{ref}/binary.html[Binary fields] are not supported in detection rule exceptions. +==== + +. Do one of the following: ++ +** To add an Endpoint exception from the rule details page: ++ +... Go to the rule details page (**Rules** → **Detection rules (SIEM)**), and then search for and select one of the <>. +... Scroll down the rule details page, select the **Endpoint exceptions** tab, then click **Add endpoint exception**. +** To add an Endpoint exception from the Alerts table: ++ +... Go to **Alerts**. +... Scroll down to the Alerts table, and from an {elastic-endpoint} +alert, click the **More actions** menu (image:images/icons/boxesHorizontal.svg[Actions menu]), then select **Add Endpoint exception**. +** To add an Endpoint exception from Shared Exception Lists page: ++ +... Go to **Rules** → **Shared exception lists**. +... Expand the Endpoint Security Exception List or click the list name to open the list's details page. Next, click **Add endpoint exception**. ++ +[NOTE] +==== +The Endpoint Security Exception List is automatically created. By default, it's associated with endpoint protection rules and any rules with the <> option selected. +==== ++ +The **Add Endpoint Exception** flyout opens. ++ +[role="screenshot"] +image::images/add-exceptions/-detections-endpoint-add-exp.png[] +. If required, modify the conditions. Refer to <> for more information on when nested conditions are required. ++ +[IMPORTANT] +==== +Rule exceptions are case-sensitive, which means that any character that's entered as an uppercase or lowercase letter will be treated as such. In the event you _don't_ want a field evaluated as case-sensitive, some ECS fields have a `.caseless` version that you can use. +==== ++ +[NOTE] +==== +* Fields with conflicts are marked with a warning icon (image:images/icons/warning.svg[Warning]). Using these fields might cause unexpected exceptions behavior. For more information, refer to <>. +* Identical, case-sensitive values are supported for the `is one of` and `is not one of` operators. For example, if you want to match the values `Windows` and `windows`, add both values to the **Value** field. +==== +. (Optional) Add a comment to the exception. +. You can select any of the following: ++ +** **Close this alert**: Closes the alert when the exception is added. This option +is only available when adding exceptions from the Alerts table. +** **Close all alerts that match this exception and were generated by this rule**: +Closes all alerts that match the exception's conditions. +. Click **Add Endpoint Exception**. An exception is created for both the detection rule and the {elastic-endpoint}. ++ +[NOTE] +==== +It might take longer for exceptions to be applied to hosts within larger deployments. +==== + +[discrete] +[[ex-nested-conditions]] +== Exceptions with nested conditions + +Some Endpoint objects contain nested fields, and the only way to ensure you are +excluding the correct fields is with nested conditions. One example is the +`process.Ext` object: + +[source,json] +---- +{ + "ancestry": [], + "code_signature": { + "trusted": true, + "subject_name": "LFC", + "exists": true, + "status": "trusted" + }, + "user": "WDAGUtilityAccount", + "token": { + "elevation": true, + "integrity_level_name": "high", + "domain": "27FB305D-3838-4", + "user": "WDAGUtilityAccount", + "elevation_type": "default", + "sid": "S-1-5-21-2047949552-857980807-821054962-504" + } +} +---- + +Only these objects require nested conditions to ensure the exception functions +correctly: + +* `Endpoint.policy.applied.artifacts.global.identifiers` +* `Endpoint.policy.applied.artifacts.user.identifiers` +* `Target.dll.Ext.code_signature` +* `Target.process.Ext.code_signature` +* `Target.process.Ext.token.privileges` +* `Target.process.parent.Ext.code_signature` +* `Target.process.thread.Ext.token.privileges` +* `dll.Ext.code_signature` +* `file.Ext.code_signature` +* `file.Ext.macro.errors` +* `file.Ext.macro.stream` +* `process.Ext.code_signature` +* `process.Ext.token.privileges` +* `process.parent.Ext.code_signature` +* `process.thread.Ext.token.privileges` + +[discrete] +[[security-add-exceptions-nested-condition-example]] +=== Nested condition example + +Creates an exception that excludes all LFC-signed trusted processes: + +[role="screenshot"] +image::images/add-exceptions/-detections-nested-exp.png[] + +[discrete] +[[manage-exception]] +== View and manage exceptions + +To view a rule's exceptions, open the rule's details page (**Rules** → **Detection rules (SIEM)** → **_Rule name_**), then scroll down and select the **Rule exceptions** or **Endpoint exceptions** tab. All exceptions that belong to the rule will display in a list. From the list, you can filter, edit, and delete exceptions. You can also toggle between **Active exceptions** and **Expired exceptions**. + +[role="screenshot"] +image::images/add-exceptions/-detections-manage-default-rule-list.png[A default rule list] + +[discrete] +[[rules-using-same-exception]] +== Find rules using the same exceptions + +To find out if an exception is used by other rules, select the **Rule exceptions** or **Endpoint exceptions** tab, navigate to an exception list item, then click **Affects _X_ rules**. + +[NOTE] +==== +Changes that you make to the exception also apply to other rules that use the exception. +==== + +[role="screenshot"] +image::images/add-exceptions/-detections-exception-affects-multiple-rules.png[Exception that affects multiple rules] diff --git a/docs/serverless/rules/detection-engine-overview.asciidoc b/docs/serverless/rules/detection-engine-overview.asciidoc new file mode 100644 index 0000000000..3d89447e5c --- /dev/null +++ b/docs/serverless/rules/detection-engine-overview.asciidoc @@ -0,0 +1,134 @@ +[[security-detection-engine-overview]] += Detection engine overview + +// :description: Learn about the detection engine and its features. +// :keywords: serverless, security, overview + + +Use the detection engine to create and manage rules and view the alerts +these rules create. Rules periodically search indices (such as `logs-*` and +`filebeat-*`) for suspicious source events and create alerts when a rule's +conditions are met. When an alert is created, its status is `Open`. To help +track investigations, an alert's <> can be set as +`Open`, `Acknowledged`, or `Closed`. + +[role="screenshot"] +image::images/detection-engine-overview/-detections-alert-page.png[Alerts page] + +In addition to creating <>, enable +<> to immediately start detecting +suspicious activity. For detailed information on all the prebuilt rules, see the <>. Once the prebuilt rules are loaded and +running, <> and <> explain +how to modify the rules to reduce false positives and get a better set of +actionable alerts. You can also use exceptions and value lists when creating or +modifying your own rules. + +There are several special prebuilt rules you need to know about: + +// Links to prebuilt rule pages temporarily removed for initial serverless docs. + +* <>: Automatically create alerts based on {elastic-defend}'s threat monitoring and prevention. + +// Links to prebuilt rule pages temporarily removed for initial serverless docs. + +* **External Alerts**: Automatically creates an alert for +all incoming third-party system alerts (for example, Suricata alerts). + +If you want to receive notifications via external systems, such as Slack or +email, when alerts are created, use the {kibana-ref}/alerting-getting-started.html[Alerting and Actions] framework. + +After rules have started running, you can monitor their executions to verify +they are functioning correctly, as well as view, manage, and troubleshoot +alerts (see <> and <>). + +You can create and manage rules and alerts via the UI or the {security-guide}/rule-api-overview.html[Detections API]. + +// Link to classic docs until serverless API docs are available. + +[IMPORTANT] +==== +To make sure you can access Detections and manage rules, see +<>. +==== + +[discrete] +[[support-indicator-rules]] +== Limited support for indicator match rules + +Indicator match rules provide a powerful capability to search your security data; however, their queries can consume significant deployment resources. When creating an <>, we recommend limiting the time range of the indicator index query to the minimum period necessary for the desired rule coverage. For example, the default indicator index query `@timestamp > "now-30d/d"` searches specified indicator indices for indicators ingested during the past 30 days and rounds the query start time down to the nearest day (resolves to UTC `00:00:00`). Without this limitation, the rule will include all of the indicators in your indicator indices, which may extend the time it takes for the indicator index query to complete. + +In addition, indicator match rules with an additional look-back time value greater than 24 hours are not supported. + +[discrete] +[[detections-permissions]] +== Detections configuration and prerequisites + +<> provides detailed information on all the +permissions required to initiate and use the Detections feature. + +[discrete] +[[malware-prevention]] +== Malware prevention + +Malware, short for malicious software, is any software program designed to damage or execute unauthorized actions on a +computer system. Examples of malware include viruses, worms, Trojan horses, adware, scareware, and spyware. Some +malware, such as viruses, can severely damage a computer's hard drive by deleting files or directory information. Other +malware, such as spyware, can obtain user data without their knowledge. + +Malware may be stealthy and appear as legitimate executable code, scripts, active content, and other software. It is also +often embedded in non-malicious files, non-suspicious websites, and standard programs — sometimes making the root +source difficult to identify. If infected and not resolved promptly, malware can cause irreparable damage to a computer +network. + +For information on how to enable malware protection on your host, see <>. + +[discrete] +[[machine-learning-model]] +=== Machine learning model + +To determine if a file is malicious or benign, a machine learning model looks for static attributes of files (without executing +the file) that include file structure, layout, and content. This includes information such as file header data, imports, exports, +section names, and file size. These attributes are extracted from millions of benign and malicious file samples, which then +are passed to a machine-learning algorithm that distinguishes a benign file from a malicious one. The machine learning +model is updated as new data is procured and analyzed. + +[discrete] +[[security-detection-engine-overview-threshold]] +=== Threshold + +A malware threshold determines the action the agent should take if malware is detected. The Elastic Agent uses a recommended threshold level that generates a balanced number of alerts with a low probability of undetected malware. This threshold also minimizes the number of false positive alerts. + +[discrete] +[[ransomware-prevention]] +== Ransomware prevention + +Ransomware is computer malware that installs discreetly on a user's computer and encrypts data until a specified amount of money (ransom) is paid. Ransomware is usually similar to other malware in its delivery and execution, infecting systems +through spear-phishing or drive-by downloads. If not resolved immediately, ransomware can cause irreparable damage to an entire computer network. + +Behavioral ransomware prevention on the Elastic Endpoint detects and stops ransomware attacks on Windows systems by analyzing data from low-level system processes, and is effective across an array of widespread ransomware families — including those targeting the system’s master boot record. + +For information on how to enable ransomware protection on your host, see <>. + +[discrete] +[[security-detection-engine-overview-resolve-ui-error-messages]] +=== Resolve UI error messages + +Depending on your user role privileges and whether detection system indices have already been created, you might get one of these error messages when you +open the **Alerts** or **Rules** page: + +* **`Let’s set up your detection engine`** ++ +If you get this message, a user with specific privileges must visit the +**Alerts** or **Rules** page before you can view detection alerts and rules. +Refer to <> for a list of all the requirements. +* **`Detection engine permissions required`** ++ +If you get this message, you do not have the +<> to view the **Detections** feature, +and you should contact your project administrator. + +[discrete] +[[detections-logsdb-index-mode]] +== Using logsdb index mode + +logsdb is enabled by default for Elastic serverless. Refer to <> to learn more. \ No newline at end of file diff --git a/docs/serverless/rules/rules-ui-create.asciidoc b/docs/serverless/rules/rules-ui-create.asciidoc new file mode 100644 index 0000000000..736ea49017 --- /dev/null +++ b/docs/serverless/rules/rules-ui-create.asciidoc @@ -0,0 +1,896 @@ +[[security-rules-create]] += Create a detection rule + +// :description: Create detection rules to monitor your environment for suspicious and malicious behavior. +// :keywords: serverless, security, defend, how-to, manage, secure + + +To create a new detection rule, follow these steps: + +. Define the <>. The configuration for this step varies depending on the rule type. +. Configure basic rule settings. +. Configure advanced rule settings (optional). +. Set the rule's schedule. +. Set up rule actions (optional). +. Set up response actions (optional). + +.Requirements +[NOTE] +==== +* To create detection rules, you must have access to data views, which requires the appropriate user role. +* You'll also need permissions to enable and view detections, manage rules, manage alerts, and preview rules. These permissions depend on the user role. Refer to <> for more information. +==== + +[TIP] +==== +At any step, you can <> before saving it to see what kind of results you can expect. +==== + +[discrete] +[[create-custom-rule]] +== Create a custom query rule + +. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. +. To create a rule based on a KQL or Lucene query, select **Custom query**, +then: ++ +.. Define which {es} indices or data view the rule searches for alerts. +.. Use the filter and query fields to create the criteria used for detecting +alerts. ++ +The following example (based on the prebuilt rule Volume Shadow Copy Deleted or Resized via VssAdmin) detects when the `vssadmin delete shadows` +Windows command is executed: ++ +*** **Index patterns**: `winlogbeat-*` ++ +Winlogbeat ships Windows event logs to {elastic-sec}. +*** **Custom query**: `event.action:"Process Create (rule: ProcessCreate)" and process.name:"vssadmin.exe" and process.args:("delete" and "shadows")` ++ +Searches the `winlogbeat-*` indices for `vssadmin.exe` executions with +the `delete` and `shadow` arguments, which are used to delete a volume's shadow +copies. ++ +[role="screenshot"] +image::images/rules-ui-create/-detections-rule-query-example.png[Rule query example] +.. You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. ++ +When you use a saved query, the **Load saved query "_query name_" dynamically on each rule execution** check box appears: ++ +*** Select this to use the saved query every time the rule runs. This links the rule to the saved query, and you won't be able to modify the rule's **Custom query** field or filters because the rule will only use settings from the saved query. To make changes, modify the saved query itself. +*** Deselect this to load the saved query as a one-time way of populating the rule's **Custom query** field and filters. This copies the settings from the saved query to the rule, so you can then further adjust the rule's query and filters as needed. If the saved query is later changed, the rule will not inherit those changes. +. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ +//// +/* The following steps are repeated across multiple rule types. If you change anything +in these steps or sub-steps, apply the change to the other rule types, too. */ +//// ++ +. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. ++ +.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. +.. Enter the field's data type. +. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. ++ +.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. +.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. +. Click **Continue** to <>. + +[discrete] +[[create-ml-rule]] +== Create a machine learning rule + +[IMPORTANT] +==== +To create or edit {ml} rules, you need an appropriate user role. Additionally, the selected {ml} job must be running for the rule to function correctly. +==== + +. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. +. To create a rule based on a {ml} anomaly threshold, select **Machine Learning**, +then select: ++ +.. The required {ml} jobs. ++ +[NOTE] +==== +If a required job isn't currently running, it will automatically start when you finish configuring and enable the rule. +==== +.. The anomaly score threshold above which alerts are created. +. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ +[NOTE] +==== +Because {ml} rules generate alerts from anomalies, which don't contain source event fields, you can only use anomaly fields when configuring alert suppression. +==== ++ +//// +/* The following steps are repeated across multiple rule types. If you change anything +in these steps or sub-steps, apply the change to the other rule types, too. */ +//// +. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. ++ +.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. +.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. +. Click **Continue** to <>. + +[discrete] +[[create-threshold-rule]] +== Create a threshold rule + +. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. +. To create a rule based on a source event field threshold, select **Threshold**, then: ++ +.. Define which {es} indices the rule analyzes for alerts. +.. Use the filter and query fields to create the criteria used for detecting +alerts. ++ +[NOTE] +==== +You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. +==== +.. Use the **Group by** and **Threshold** fields to determine which source event field is used as a threshold and the threshold's value. +.. Use the **Count** field to limit alerts by cardinality of a certain field. ++ +For example, if **Group by** is `source.ip, destination.ip` and its **Threshold** is `10`, an alert is generated for every pair of source and destination IP addresses that appear in at least 10 of the rule's search results. ++ +You can also leave the **Group by** field undefined. The rule then creates an alert when the number of search results is equal to or greater than the threshold value. If you set **Count** to limit the results by `process.name` >= 2, an alert will only be generated for source/destination IP pairs that appear with at least 2 unique process names across all events. ++ +[IMPORTANT] +==== +Alerts created by threshold rules are synthetic alerts that do not resemble the source documents. The alert itself only contains data about the fields that were aggregated over (the **Group by** fields). Other fields are omitted, because they can vary across all source documents that were counted toward the threshold. Additionally, you can reference the actual count of documents that exceeded the threshold from the `kibana.alert.threshold_result.count` field. +==== +. (Optional) Select **Suppress alerts** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ +//// +/* The following steps are repeated across multiple rule types. If you change anything +in these steps or sub-steps, apply the change to the other rule types, too. */ +//// +. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. ++ +.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. +.. Enter the field's data type. +. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. ++ +.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. +.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. +. Click **Continue** to <>. + +[discrete] +[[create-eql-rule]] +== Create an event correlation rule + +. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. +. To create an event correlation rule using EQL, select **Event Correlation**, then: ++ +.. Define which {es} indices or data view the rule searches when querying for events. +.. Write an {ref}/eql-syntax.html[EQL query] that searches for matching events or a series of matching events. ++ +[TIP] +==== +To find events that are missing in a sequence, use the {ref}/eql-syntax.html#eql-missing-events[missing events] syntax. +==== ++ +For example, the following rule detects when `msxsl.exe` makes an outbound +network connection: ++ +*** **Index patterns**: `winlogbeat-*` ++ +Winlogbeat ships Windows events to {elastic-sec}. +*** **EQL query**: ++ +[source,eql] +---- +sequence by process.entity_id + [process + where event.type in ("start", "process_started") + and process.name == "msxsl.exe"] + [network + where event.type == "connection" + and process.name == "msxsl.exe" + and network.direction == "outgoing"] +---- ++ +Searches the `winlogbeat-*` indices for sequences of a `msxsl.exe` process start +event followed by an outbound network connection event that was started by the +`msxsl.exe` process. ++ +[role="screenshot"] +image::images/rules-ui-create/-detections-eql-rule-query-example.png[] ++ +[NOTE] +==== +For sequence events, the {security-app} generates a single alert when all events listed in the sequence are detected. To see the matched sequence events in more detail, you can view the alert in the Timeline, and, if all events came from the same process, open the alert in Analyze Event view. +==== +. (Optional) Click the EQL settings icon (image:images/icons/controlsVertical.svg[EQL settings]) to configure additional fields used by {ref}/eql.html#specify-a-timestamp-or-event-category-field[EQL search]: ++ +** **Event category field**: Contains the event classification, such as `process`, `file`, or `network`. This field is typically mapped as a field type in the {ref}/keyword.html[keyword family]. Defaults to the `event.category` ECS field. +** **Tiebreaker field**: Sets a secondary field for sorting events (in ascending, lexicographic order) if they have the same timestamp. +** **Timestamp field**: Contains the event timestamp used for sorting a sequence of events. This is different from the **Timestamp override** advanced setting, which is used for querying events within a range. Defaults to the `@timestamp` ECS field. +. preview:[] (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ +//// +/* The following steps are repeated across multiple rule types. If you change anything +in these steps or sub-steps, apply the change to the other rule types, too. */ +//// +. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. ++ +.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. +.. Enter the field's data type. +. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. ++ +.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. +.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. +. Click **Continue** to <>. + +[discrete] +[[create-indicator-rule]] +== Create an indicator match rule + +[NOTE] +==== +{elastic-sec} provides limited support for indicator match rules. See <> for more information. +==== + +. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. +. To create a rule that searches for events whose specified field value matches the specified indicator field value in the indicator index patterns, select **Indicator Match**, then fill in the following fields: ++ +.. **Source**: The individual index patterns or data view that specifies what data to search. +.. **Custom query**: The query and filters used to retrieve the required results from +the {elastic-sec} event indices. For example, if you want to match documents that only contain a `destination.ip` address field, add `destination.ip : *`. ++ +[TIP] +==== +If you want the rule to check every field in the indices, use this +wildcard expression: `*:*`. +==== ++ +[NOTE] +==== +You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. +==== +.. **Indicator index patterns**: The indicator index patterns containing field values for which you want to generate alerts. This field is automatically populated with indices specified in the `securitySolution:defaultThreatIndex` advanced setting. For more information, see <>. ++ +[IMPORTANT] +==== +Data in indicator indices must be <>, and so it must contain a `@timestamp` field. +==== +.. **Indicator index query**: The query and filters used to filter the fields from +the indicator index patterns. The default query `@timestamp > "now-30d/d"` searches specified indicator indices for indicators ingested during the past 30 days and rounds the start time down to the nearest day (resolves to UTC `00:00:00`). +.. **Indicator mapping**: Compares the values of the specified event and indicator fields, and generates an alert if the values are identical. ++ +[NOTE] +==== +Only single-value fields are supported. +==== ++ +To define which field values are compared from the indices, add the following: ++ +*** **Field**: The field used for comparing values in the {elastic-sec} event +indices. +*** **Indicator index field**: The field used for comparing values in the indicator +indices. +.. You can add `AND` and `OR` clauses to define when alerts are generated. ++ +For example, to create a rule that generates alerts when `host.name` **and** +`destination.ip` field values in the `logs-*` or `packetbeat-*` {elastic-sec} indices +are identical to the corresponding field values in the `mock-threat-list` indicator +index, enter the rule parameters seen in the following image: ++ +[role="screenshot"] +image::images/rules-ui-create/-detections-indicator-rule-example.png[Indicator match rule settings] ++ +[TIP] +==== +Before you create rules, create <> so you can select them under **Timeline template** at the end of the **Define rule** section. When alerts generated by the rule are investigated in the Timeline, Timeline query values are replaced with their corresponding alert field values. +==== +. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ +//// +/* The following steps are repeated across multiple rule types. If you change anything +in these steps or sub-steps, apply the change to the other rule types, too. */ +//// +. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. ++ +.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. +.. Enter the field's data type. +. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. ++ +.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. +.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. +. Click **Continue** to <>. + +[discrete] +[[indicator-value-lists]] +=== Use value lists with indicator match rules + +While there are numerous ways you can add data into indicator indices, you can use value lists as the indicator match index in an indicator match rule. Take the following scenario, for example: + +You uploaded a value list of known ransomware domains, and you want to be notified if any of those domains matches a value contained in a domain field in your security event index pattern. + +. Upload a value list of indicators. +. Create an indicator match rule and fill in the following fields: ++ +.. **Index patterns**: The Elastic Security event indices on which the rule runs. +.. **Custom query**: The query and filters used to retrieve the required results from the Elastic Security event indices (e.g., `host.domain :*`). +.. **Indicator index patterns**: Value lists are stored in a hidden index called `.items-`. Enter the name of the {kib} space in which this rule will run in this field. +.. **Indicator index query**: Enter the value `list_id :`, followed by the name of the value list you want to use as your indicator index (uploaded in Step 1 above). +.. **Indicator mapping** ++ +*** **Field**: Enter the field from the Elastic Security event indices to be used for comparing values. +*** **Indicator index field**: Enter the type of value list you created (i.e., `keyword`, `text`, or `IP`). ++ +[TIP] +==== +If you don't remember this information, go to **Rules** → **Detection rules (SIEM)** → **Manage value lists**. Locate the appropriate value list and note the field in the corresponding `Type` column. (Examples include keyword, text, and IP.) +==== + +[role="screenshot"] +image::images/rules-ui-create/-detections-indicator_value_list.png[] + +[discrete] +[[create-new-terms-rule]] +== Create a new terms rule + +. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. +. To create a rule that searches for each new term detected in source documents, select **New Terms**, then: ++ +.. Specify what data to search by entering individual {es} index patterns or selecting an existing data view. +.. Use the filter and query fields to create the criteria used for detecting +alerts. ++ +[NOTE] +==== +You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. +==== +.. Use the **Fields** menu to select a field to check for new terms. You can also select up to three fields to detect a combination of new terms (for example, a `host.ip` and `host.id` that have never been observed together before). ++ +[IMPORTANT] +==== +When checking multiple fields, each unique combination of values from those fields is evaluated separately. For example, a document with `host.name: ["host-1", "host-2", "host-3"]` and `user.name: ["user-1", "user-2", "user-3"]` has 9 (3x3) unique combinations of `host.name` and `user.name`. A document with 11 values in `host.name` and 10 values in `user.name` has 110 (11x10) unique combinations. The new terms rule only evaluates 100 unique combinations per document, so selecting fields with large arrays of values might cause incorrect results. +==== +.. Use the **History Window Size** menu to specify the time range to search in minutes, hours, or days to determine if a term is new. The history window size must be larger than the rule interval plus additional look-back time, because the rule will look for terms where the only time(s) the term appears within the history window is _also_ within the rule interval and additional look-back time. ++ +For example, if a rule has an interval of 5 minutes, no additional look-back time, and a history window size of 7 days, a term will be considered new only if the time it appears within the last 7 days is also within the last 5 minutes. Configure the rule interval and additional look-back time when you <>. +. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ +//// +/* The following steps are repeated across multiple rule types. If you change anything +in these steps or sub-steps, apply the change to the other rule types, too. */ +//// +. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. ++ +.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. +.. Enter the field's data type. +. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. ++ +.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. +.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. +. Click **Continue** to <>. + +[discrete] +[[create-esql-rule]] +== Create an {esql} rule + +Use {ref}/esql.html[{esql}] to query your source events and aggregate event data. Query results are returned in a table with rows and columns. Each row becomes an alert. + +To create an {esql} rule: + +. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page appears. +. Select **{esql}**, then write a query. ++ +[NOTE] +==== +Refer to the sections below to learn more about <>, <>, and <>. +==== ++ +[TIP] +==== +Click the help icon (image:images/icons/iInCircle.svg[Click the ES|QL help icon]) to open the in-product reference documentation for all {esql} commands and functions. +==== +. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ +//// +/* The following steps are repeated across multiple rule types. If you change anything +in these steps or sub-steps, apply the change to the other rule types, too. */ +//// +. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. ++ +.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. +.. Enter the field's data type. +. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. ++ +.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. +.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. +. Click **Continue** to <>. + +[discrete] +[[esql-rule-query-types]] +=== {esql} query types + +{esql} rule queries are loosely categorized into two types: aggregating and non-aggregating. + +[discrete] +[[esql-agg-query]] +==== Aggregating query + +Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and any new fields that the query creates. + +[NOTE] +==== +A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the `STATS...BY` function to create a column with aggregated values, the column is created when the rule runs and is added as a new field to any alerts that are generated by the rule. +==== + +Here is an example aggregating query: + +[source,esql] +---- +FROM logs-* +| STATS host_count = COUNT(host.name) BY host.name +| SORT host_count DESC +| WHERE host_count > 20 +---- + +* This query starts by searching logs from indices that match the pattern `logs-*`. +* The query then aggregates the count of events by `host.name`. +* Next, it sorts the result by `host_count` in descending order. +* Then, it filters for events where the `host_count` field appears more than 20 times during the specified rule interval. + +[NOTE] +==== +Rules that use aggregating queries might create duplicate alerts. This can happen when events that occur in the additional look-back time are aggregated both in the current rule execution and in a previous rule execution. +==== + +[discrete] +[[esql-non-agg-query]] +==== Non-aggregating query + +Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. + +[NOTE] +==== +A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the columns are created when the rule runs and are added as new fields to any alerts generated by the rule. +==== + +Here is an example non-aggregating query: + +[source,esql] +---- +FROM logs-* METADATA _id, _index, _version +| WHERE event.category == "process" AND event.id == "8a4f500d" +| LIMIT 10 +---- + +* This query starts by querying logs from indices that match the pattern `logs-*`. The `METADATA _id, _index, _version` operator allows <>. +* Next, the query filters events where the `event.category` is a process and the `event.id` is `8a4f500d`. +* Then, it limits the output to the top 10 results. + +[discrete] +[[esql-non-agg-query-dedupe]] +==== Turn on alert deduplication for rules using non-aggregating queries + +To deduplicate alerts, a query needs access to the `_id`, `_index`, and `_version` metadata fields of the queried source event documents. You can allow this by adding the `METADATA _id, _index, _version` operator after the `FROM` source command, for example: + +[source,esql] +---- +FROM logs-* METADATA _id, _index, _version +| WHERE event.category == "process" AND event.id == "8a4f500d" +| LIMIT 10 +---- + +When those metadata fields are provided, unique alert IDs are created for each alert generated by the query. + +When developing the query, make sure you don't {ref}/esql-commands.html#esql-drop[`DROP`] or filter out the `_id`, `_index`, or `_version` metadata fields. + +Here is an example of a query that fails to deduplicate alerts. It uses the `DROP` command to omit the `_id` property from the results table: + +[source,esql] +---- +FROM logs-* METADATA _id, _index, _version +| WHERE event.category == "process" AND event.id == "8a4f500d" +| DROP _id +| LIMIT 10 +---- + +Here is another example of an invalid query that uses the `KEEP` command to only return `event.*` fields in the results table: + +[source,esql] +---- +FROM logs-* METADATA _id, _index, _version +| WHERE event.category == "process" AND event.id == "8a4f500d" +| KEEP event.* +| LIMIT 10 +---- + +[discrete] +[[esql-query-design]] +=== Query design considerations + +When writing your query, consider the following: + +* The {ref}/esql-commands.html#esql-limit[`LIMIT`] command specifies the maximum number of rows an {esql} query returns and the maximum number of alerts created per rule run. Similarly, a detection rule's **Max alerts per run** setting specifies the maximum number of alerts it can create every time it runs. ++ +If the `LIMIT` value and **Max alerts per run** value are different, the rule uses the lower value to determine the maximum number of alerts the rule generates. +* When writing an aggregating query, use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] command with fields that you want to search and filter for after alerts are created. For example, using the `host.name`, `user.name`, `process.name` fields with the `BY` operator of the `STATS...BY` command returns these fields in alert documents, and allows you to search and filter for them from the Alerts table. +* When configuring alert suppression on a non-aggregating query, we recommend sorting results by ascending `@timestamp` order. Doing so ensures that alerts are properly suppressed, especially if the number of alerts generated is higher than the **Max alerts per run** value. + +[discrete] +[[esql-rule-limitations]] +=== {esql} rule limitations + +If your {esql} query creates new fields that aren’t part of the ECS schema, they aren't mapped to the alerts index, so you can't search for or filter them in the Alerts table. As a workaround, create <>. + +[discrete] +[[custom-highlighted-esql-fields]] +=== Highlight fields returned by the {esql} rule query + +When configuring an {esql} rule's **<>**, you can specify any fields that the rule's aggregating or non-aggregating query return. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. + +[discrete] +[[rule-ui-basic-params]] +== Configure basic rule settings + +. In the **About rule** pane, fill in the following fields: ++ +.. **Name**: The rule's name. +.. **Description**: A description of what the rule does. +.. **Default severity**: Select the severity level of alerts created by the rule: ++ +*** **Low**: Alerts that are of interest but generally are not considered to be security incidents. Sometimes a combination of low severity alerts can indicate suspicious activity. +*** **Medium**: Alerts that require investigation. +*** **High**: Alerts that require an immediate investigation. +*** **Critical**: Alerts that indicate it is highly likely a security incident has occurred. +.. **Severity override** (optional): Select to use source event values to +override the **Default severity** in generated alerts. When selected, a UI +component is displayed where you can map the source event field values to +severity levels. The following example shows how to map severity levels to `host.name` +values: ++ +[role="screenshot"] +image::images/rules-ui-create/-detections-severity-mapping-ui.png[] ++ +[NOTE] +==== +For threshold rules, not all source event values can be used for overrides; only the fields that were aggregated over (the `Group by` fields) will contain data. Please also note that overrides are not supported for event correlation rules. +==== +.. **Default risk score**: A numerical value between 0 and 100 that indicates the risk of events detected by the rule. This setting changes to a default value when you change the **Severity** level, but you can adjust the risk score as needed. General guidelines are: ++ +*** `0` - `21` represents low severity. +*** `22` - `47` represents medium severity. +*** `48` - `73` represents high severity. +*** `74` - `100` represents critical severity. +.. **Risk score override** (optional): Select to use a source event value to +override the **Default risk score** in generated alerts. When selected, a UI +component is displayed to select the source field used for the risk +score. For example, if you want to use the source event's risk score in +alerts: ++ +[role="screenshot"] +image::images/rules-ui-create/-detections-risk-source-field-ui.png[] ++ +[NOTE] +==== +For threshold rules, not all source event values can be used for overrides; only the fields that were aggregated over (the `Group by` fields) will contain data. +==== +.. **Tags** (optional): Words and phrases used to categorize, filter, and search +the rule. +. Continue with **one** of the following: ++ +** <> +** <> + +[discrete] +[[rule-ui-advanced-params]] +== Configure advanced rule settings (optional) + +. Click **Advanced settings** and fill in the following fields where applicable: ++ +.. **Reference URLs** (optional): References to information that is relevant to +the rule. For example, links to background information. +.. **False positive examples** (optional): List of common scenarios that may produce +false-positive alerts. +.. **MITRE ATT&CK^TM^ threats** (optional): Add relevant https://attack.mitre.org/[MITRE] framework tactics, techniques, and subtechniques. +.. **Custom highlighted fields** (optional): Specify highlighted fields for unique alert investigation flows. You can choose any fields that are available in the you selected for the rule's data source. ++ +After you create the rule, you can find all custom highlighted fields in the About section of the rule details page. If the rule has alerts, you can find custom highlighted fields in the <> section of the alert details flyout. +.. **Setup guide** (optional): Instructions on rule prerequisites such as required integrations, configuration steps, and anything else needed for the rule to work correctly. +.. **Investigation guide** (optional): Information for analysts investigating +alerts created by the rule. You can also add action buttons to <> or <> using alert data. +.. **Author** (optional): The rule's authors. +.. **License** (optional): The rule's license. +.. **Elastic endpoint exceptions** (optional): Adds all <> to this rule. ++ +[NOTE] +==== +If you select this option, you can add {elastic-endpoint} exceptions on the Rule details page. Additionally, all future exceptions added to <> will also affect this rule. +==== +.. **Building block** (optional): Select to create a building-block rule. By default, alerts generated from a building-block rule are not displayed in the UI. See <> for more information. +.. **Max alerts per run** (optional): Specify the maximum number of alerts the rule can create each time it runs. Default is 100. +.. **Indicator prefix override**: Define the location of indicator data within the structure of indicator documents. When the indicator match rule executes, it queries specified indicator indices and references this setting to locate fields with indicator data. This data is used to enrich indicator match alerts with metadata about matched threat indicators. The default value for this setting is `threat.indicator`. ++ +[IMPORTANT] +==== +If your threat indicator data is at a different location, update this setting accordingly to ensure alert enrichment can still be performed. +==== +.. **Rule name override** (optional): Select a source event field to use as the +rule name in the UI (Alerts table). This is useful for exposing, at a glance, +more information about an alert. For example, if the rule generates alerts from +Suricata, selecting `event.action` lets you see what action (Suricata category) +caused the event directly in the Alerts table. ++ +[NOTE] +==== +For threshold rules, not all source event values can be used for overrides; only the fields that were aggregated over (the `Group by` fields) will contain data. +==== +.. **Timestamp override** (optional): Select a source event timestamp field. When selected, the rule's query uses the selected field, instead of the default `@timestamp` field, to search for alerts. This can help reduce missing alerts due to network or server outages. Specifically, if your ingest pipeline adds a timestamp when events are sent to {es}, this avoids missing alerts due to ingestion delays. +However, if you know your data source has an inaccurate `@timestamp` value, it is recommended you select the **Do not use @timestamp as a fallback timestamp field** option to ignore the `@timestamp` field entirely. ++ +[TIP] +==== +The {filebeat-ref}/filebeat-module-microsoft.html[Microsoft] and +{filebeat-ref}/filebeat-module-google_workspace.html[Google Workspace] {filebeat} modules have an `event.ingested` timestamp field that can be used instead of the default `@timestamp` field. +==== +. Click **Continue**. The **Schedule rule** pane is displayed. ++ +[role="screenshot"] +image::images/rules-ui-create/-detections-schedule-rule.png[] +. Continue with <>. + +[discrete] +[[rule-schedule]] +== Set the rule's schedule + +. Select how often the rule runs. +. Optionally, add `Additional look-back time` to the rule. When defined, the +rule searches indices with the additional time. ++ +For example, if you set a rule to run every 5 minutes with an additional +look-back time of 1 minute, the rule runs every 5 minutes but analyzes the +documents added to indices during the last 6 minutes. ++ +[IMPORTANT] +==== +It is recommended to set the `Additional look-back time` to at +least 1 minute. This ensures there are no missing alerts when a rule does not +run exactly at its scheduled time. + +{elastic-sec} prevents duplication. Any duplicate alerts that are discovered during the +`Additional look-back time` are _not_ created. +==== +. Click **Continue**. The **Rule actions** pane is displayed. +. Do either of the following: ++ +** Continue onto <> and <> (optional). +** Create the rule (with or without activation). + +[discrete] +[[rule-notifications]] +== Set up rule actions (optional) + +Use actions to set up notifications sent via other systems when alerts are generated. + +[NOTE] +==== +To use actions for alert notifications, you need the appropriate user role. For more information, see <>. +==== + +. Select a connector type to determine how notifications are sent. For example, if you select the {jira} connector, notifications are sent to your {jira} system. ++ +[NOTE] +==== +Each action type requires a connector. Connectors store the +information required to send the notification from the external system. You can +configure connectors while creating the rule or in **Project settings** → **Stack Management** → **{connectors-ui}**. For more +information, see {kibana-ref}/action-types.html[Action and connector types]. + +Some connectors that perform actions require less configuration. For example, you do not need to set the action frequency or variables for the {kibana-ref}/cases-action-type.html[Cases connector]. +==== +. After you select a connector, set its action frequency to define when notifications are sent: ++ +** **Summary of alerts**: Select this option to get a report that summarizes generated alerts, which you can review at your convenience. Alert summaries will be sent at the specified time intervals. ++ +[NOTE] +==== +When setting a custom notification frequency, do not choose a time that is shorter than the rule's execution schedule. +==== +** **For each alert**: Select this option to ensure notifications are sent every time new alerts are generated. +. (Optional) Specify additional conditions that need to be met for notifications to send. Click the toggle to enable a setting, then add the required details: ++ +** **If alert matches query**: Enter a KQL query that defines field-value pairs or query conditions that must be met for notifications to send. The query only searches alert documents in the indices specified for the rule. +** **If alert is generated during timeframe**: Set timeframe details. Notifications are only sent if alerts are generated within the timeframe you define. +. Complete the required connector type fields. Here is an example with {jira}: ++ +[role="screenshot"] +image::images/rules-ui-create/-detections-selected-action-type.png[] +. Use the default notification message or customize it. You can add more context to the message by clicking the icon above the message text box and selecting from a list of available <>. +. Create the rule with or without activation. ++ +[NOTE] +==== +When you activate a rule, it is queued, and its schedule is determined by +its initial run time. For example, if you activate a rule that runs every 5 +minutes at 14:03 but it does not run until 14:04, it will run again at 14:09. +==== + +[IMPORTANT] +==== +After you activate a rule, you can check if it is running as expected +using the <> on the Rules page. If you see +values in the `Gap` column, you can <>. + +When a rule fails to run, the {security-app} tries to rerun it at its next +scheduled run time. +==== + +[discrete] +[[rule-action-variables]] +=== Alert notification placeholders + +You can use http://mustache.github.io/[mustache syntax] to add variables to notification messages. The action frequency you choose determines the variables you can select from. + +The following variables can be passed for all rules: + +[NOTE] +==== +Refer to {kibana-ref}/rule-action-variables.html#alert-summary-action-variables[Action frequency: Summary of alerts] to learn about additional variables that can be passed if the rule's action frequency is **Summary of alerts**. +==== + +* `{{context.alerts}}`: Array of detected alerts +* `{{{context.results_link}}}`: URL to the alerts +* `{{context.rule.anomaly_threshold}}`: Anomaly threshold score above which +alerts are generated ({ml} rules only) +* `{{context.rule.description}}`: Rule description +* `{{context.rule.false_positives}}`: Rule false positives +* `{{context.rule.filters}}`: Rule filters (query rules only) +* `{{context.rule.id}}`: Unique rule ID returned after creating the rule +* `{{context.rule.index}}`: Indices rule runs on (query rules only) +* `{{context.rule.language}}`: Rule query language (query rules only) +* `{{context.rule.machine_learning_job_id}}`: ID of associated {ml} job ({ml} +rules only) +* `{{context.rule.max_signals}}`: Maximum allowed number of alerts per rule +execution +* `{{context.rule.name}}`: Rule name +* `{{context.rule.query}}`: Rule query (query rules only) +* `{{context.rule.references}}`: Rule references +* `{{context.rule.risk_score}}`: Default rule risk score ++ +[NOTE] +==== +This placeholder contains the rule's default values even when the **Risk score override** option is used. +==== +* `{{context.rule.rule_id}}`: Generated or user-defined rule ID that can be +used as an identifier across systems +* `{{context.rule.saved_id}}`: Saved search ID +* `{{context.rule.severity}}`: Default rule severity ++ +[NOTE] +==== +This placeholder contains the rule's default values even when the **Severity override** option is used. +==== +* `{{context.rule.threat}}`: Rule threat framework +* `{{context.rule.threshold}}`: Rule threshold values (threshold rules only) +* `{{context.rule.timeline_id}}`: Associated Timeline ID +* `{{context.rule.timeline_title}}`: Associated Timeline name +* `{{context.rule.type}}`: Rule type +* `{{context.rule.version}}`: Rule version +* `{{date}}`: Date the rule scheduled the action +* `{{kibanaBaseUrl}}`: Configured `server.publicBaseUrl` value, or empty string if not configured +* `{{rule.id}}`: ID of the rule +* `{{rule.name}}`: Name of the rule +* `{{rule.spaceId}}`: Space ID of the rule +* `{{rule.tags}}`: Tags of the rule +* `{{rule.type}}`: Type of rule +* `{{state.signals_count}}`: Number of alerts detected + +The following variables can only be passed if the rule’s action frequency is for each alert: + +* `{{alert.actionGroup}}`: Action group of the alert that scheduled actions for the rule +* `{{alert.actionGroupName}}`: Human-readable name of the action group of the alert that scheduled actions for the rule +* `{{alert.actionSubgroup}}`: Action subgroup of the alert that scheduled actions for the rule +* `{{alert.id}}`: ID of the alert that scheduled actions for the rule +* `{{alert.flapping}}`: A flag on the alert that indicates whether the alert status is changing repeatedly + +[discrete] +[[placeholder-examples]] +==== Alert placeholder examples + +To understand which fields to parse, see the {security-guide}/rule-api-overview.html[Detections API] to view the JSON representation of rules. + +// Link to classic docs until serverless API docs are available. + +Example using `{{context.rule.filters}}` to output a list of filters: + +[source,json] +---- +{{#context.rule.filters}} +{{^meta.disabled}}{{meta.key}} {{#meta.negate}}NOT {{/meta.negate}}{{meta.type}} {{^exists}}{{meta.value}}{{meta.params.query}}{{/exists}}{{/meta.disabled}} +{{/context.rule.filters}} +---- + +Example using `{{context.alerts}}` as an array, which contains each alert generated since the last time the action was executed: + +[source,json] +---- +{{#context.alerts}} +Detection alert for user: {{user.name}} +{{/context.alerts}} +---- + +Example using the mustache "current element" notation `{{.}}` to output all the rule references in the `signal.rule.references` array: + +[source,json] +---- +{{#signal.rule.references}} {{.}} {{/signal.rule.references}} +---- + +[discrete] +[[rule-response-action]] +=== Set up response actions (optional) + +Use response actions to set up additional functionality that will run whenever a rule executes: + +* **Osquery**: Include live Osquery queries with a custom query rule. When an alert is generated, Osquery automatically collects data on the system related to the alert. Refer to <> to learn more. +* **{elastic-defend}**: Automatically run response actions on an endpoint when rule conditions are met. For example, you can automatically isolate a host or terminate a process when specific activities or events are detected on the host. Refer to <> to learn more. + +[IMPORTANT] +==== +Host isolation involves quarantining a host from the network to prevent further spread of threats and limit potential damage. Be aware that automatic host isolation can cause unintended consequences, such as disrupting legitimate user activities or blocking critical business processes. +==== + +[discrete] +[[preview-rules]] +== Preview your rule (optional) + +You can preview any custom or prebuilt rule to find out how noisy it will be. For a custom rule, you can then adjust the rule's query or other settings. + +[NOTE] +==== +To preview rules, you must have the appropriate user role. Refer to <> for more information. +==== + +Click the **Rule preview** button while creating or editing a rule. The preview opens in a side panel, showing a histogram and table with the alerts you can expect, based on the defined rule settings and past events in your indices. + +[role="screenshot"] +image::images/rules-ui-create/-detections-preview-rule.png[Rule preview] + +The preview also includes the effects of rule exceptions and override fields. In the histogram, alerts are stacked by `event.category` (or `host.name` for machine learning rules), and alerts with multiple values are counted more than once. + +To interact with the rule preview: + +* Use the date and time picker to define the preview's time range. ++ +[TIP] +==== +Avoid setting long time ranges with short rule intervals, or the rule preview might time out. +==== +* Click **Refresh** to update the preview. ++ +** When you edit the rule's settings or the preview's time range, the button changes from blue to green to indicate that the rule has been edited since the last preview. +** For a relative time range (such as `Last 1 hour`), refresh the preview to check for the latest results. (Previews don't automatically refresh with new incoming data.) +* Click the **View details** icon (image:images/icons/expand.svg[View details]) in the alerts table to view the details of a particular alert. +* To resize the preview, hover between the rule settings and preview, then click and drag the border. You can also click the border, then the collapse icon (image:images/icons/menuRight.svg[Collapse menu]) to collapse and expand the preview. +* To close the preview, click the **Rule preview** button again. + +[discrete] +[[view-rule-es-queries]] +=== View your rule's {es} queries (optional) + +[NOTE] +==== +This option is only offered for {esql} and event correlation rules. +==== + +When previewing a rule, you can also learn about its {es} queries, which are submitted when the rule runs. This information can help you identify and troubleshoot potential rule issues. You can also use it to confirm that your rule is retrieving the expected data. + +To learn more about your rule's {es} queries, preview its results and do the following: + +. Select the **Show {es} requests, ran during rule executions** option below the preview's date and time picker. The **Preview logged results** section displays under the histogram and alerts table. +. Click the **Preview logged results** section to expand it. Within the section, each rule execution is shown on an individual row. +. Expand each row to learn more about the {es} queries that the rule submits each time it executes. The following details are provided: ++ +** When it started, and how long it took to complete +** A brief explanation of what the {es} queries do +** The actual {es} queries that the rule submits to indices containing events that are used during the rule execution ++ +[TIP] +==== +Run the queries in <> to determine if your rule is retrieving the expected data. For example, to test your rule’s exceptions, run the rule’s {es} queries, which will also contain exceptions added to the rule. If your rule’s exceptions are working as intended, the query will not return events that should be ignored. +==== From fc161af44e0cd6b052eada8cf9acc9caadb40297 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Wed, 8 Jan 2025 15:01:45 +0000 Subject: [PATCH 2/2] Delete docs/serverless directory and its contents --- .../endpoint-protection-rules.asciidoc | 43 - docs/serverless/index.asciidoc | 201 ---- docs/serverless/rules/add-exceptions.asciidoc | 285 ------ .../rules/detection-engine-overview.asciidoc | 134 --- .../serverless/rules/rules-ui-create.asciidoc | 896 ------------------ 5 files changed, 1559 deletions(-) delete mode 100644 docs/serverless/edr-manage/endpoint-protection-rules.asciidoc delete mode 100644 docs/serverless/index.asciidoc delete mode 100644 docs/serverless/rules/add-exceptions.asciidoc delete mode 100644 docs/serverless/rules/detection-engine-overview.asciidoc delete mode 100644 docs/serverless/rules/rules-ui-create.asciidoc diff --git a/docs/serverless/edr-manage/endpoint-protection-rules.asciidoc b/docs/serverless/edr-manage/endpoint-protection-rules.asciidoc deleted file mode 100644 index f71d61c72f..0000000000 --- a/docs/serverless/edr-manage/endpoint-protection-rules.asciidoc +++ /dev/null @@ -1,43 +0,0 @@ -[[endpoint-protection-rules]] -= Endpoint protection rules - -Endpoint protection rules are <> designed to help you manage and respond to alerts generated by {elastic-endpoint}, the installed component that performs {elastic-defend}'s threat monitoring and prevention. These rules include the Endpoint Security rule as well as additional detection and prevention rules for different {elastic-defend} protection features. - -IMPORTANT: To receive {elastic-endpoint} alerts, you must install {agent} and the {elastic-defend} integration on your hosts (refer to <>). - -When endpoint protection rules are triggered, {elastic-endpoint} alerts are displayed as detection alerts in the {security-app}. The detection alert name is taken from the {elastic-endpoint} alert message and overwrites the prebuilt rule name in the Alerts table. For example, for malware protection, the following {elastic-endpoint} alerts are displayed as detection alerts: - -** Malware Prevention Alert -** Malware Detection Alert - -[discrete] -[[endpoint-sec-rule]] -== Endpoint Security rule - -The Endpoint Security rule automatically creates an alert from all incoming {elastic-endpoint} alerts. - -NOTE: When you install Elastic prebuilt rules, the Endpoint Security rule that is enabled by default. - -[discrete] -[[feature-protection-rules]] -== Feature-specific protection rules - -The following endpoint protection rules give you more granular control over how you handle the generated alerts. These rules are tailored for each of {elastic-defend}'s endpoint protection features—malware, ransomware, memory threats, and malicious behavior. Enabling these rules allows you to configure more specific actions based on the protection feature and whether the malicious activity was prevented or detected. - -* Behavior - Detected - Elastic Defend -* Behavior - Prevented - Endpoint Defend -* Malicious File - Detected - Elastic Defend -* Malicious File - Prevented - Elastic Defend -* Memory Signature - Detected - Elastic Defend -* Memory Signature - Prevented - Elastic Defend -* Ransomware - Detected - Elastic Defend -* Ransomware - Prevented - Elastic Defend - -NOTE: If you choose to use the feature-specific protection rules, we recommend that you disable the Endpoint Security rule, as using both will result in duplicate alerts. - -To use these rules, you need to manually enable them from the **Rules** page in the {security-app}. Follow the instructions for <>. - -[discrete] -== Endpoint security exception handling - -All endpoint protection rules share a common exception list called the Endpoint Security Exception List. This ensures that if you switch between using the Endpoint Security rule and the feature-specific protection rules, your existing <> continue to apply. \ No newline at end of file diff --git a/docs/serverless/index.asciidoc b/docs/serverless/index.asciidoc deleted file mode 100644 index 83e32bbd4e..0000000000 --- a/docs/serverless/index.asciidoc +++ /dev/null @@ -1,201 +0,0 @@ -:doctype: book - -include::{asciidoc-dir}/../../shared/versions/stack/master.asciidoc[] -include::{asciidoc-dir}/../../shared/attributes.asciidoc[] - -[[what-is-security-serverless]] -== {sec-serverless} - -++++ -Elastic Security -++++ - -include::./what-is-security-serverless.asciidoc[leveloffset=+2] - -include::./security-overview.asciidoc[leveloffset=+2] - -include::./billing.asciidoc[leveloffset=+2] - -include::./projects-create/create-project.asciidoc[leveloffset=+2] - -include::./sec-requirements.asciidoc[leveloffset=+2] - -include::./security-ui.asciidoc[leveloffset=+2] -include::./security-spaces.asciidoc[leveloffset=+3] - -include::./AI-for-security/ai-for-security-landing-pg.asciidoc[leveloffset=+2] -include::./AI-for-security/ai-assistant.asciidoc[leveloffset=+3] -include::./AI-for-security/knowledge-base.asciidoc[leveloffset=+4] -include::./AI-for-security/attack-discovery.asciidoc[leveloffset=+3] -include::./AI-for-security/llm-connector-guides.asciidoc[leveloffset=+3] -include::./AI-for-security/llm-performance-matrix.asciidoc[leveloffset=+4] -include::./AI-for-security/connect-to-azure-openai.asciidoc[leveloffset=+4] -include::./AI-for-security/connect-to-bedrock.asciidoc[leveloffset=+4] -include::./AI-for-security/connect-to-openai.asciidoc[leveloffset=+4] -include::./AI-for-security/connect-to-vertex.asciidoc[leveloffset=+4] -include::./AI-for-security/connect-to-byo-llm.asciidoc[leveloffset=+4] -include::./AI-for-security/ai-use-cases.asciidoc[leveloffset=+3] -include::./AI-for-security/usecase-attack-disc-ai-assistant-incident-reporting.asciidoc[leveloffset=+4] -include::./AI-for-security/ai-assistant-alert-triage.asciidoc[leveloffset=+4] -include::./AI-for-security/ai-assistant-esql-queries.asciidoc[leveloffset=+4] - -include::./ingest/ingest-data.asciidoc[leveloffset=+2] -include::./ingest/threat-intelligence.asciidoc[leveloffset=+3] -include::./ingest/auto-import.asciidoc[leveloffset=+3] -include::./ingest/agentless-integrations.asciidoc[leveloffset=+3] -include::./ingest/agentless-troubleshooting.asciidoc[leveloffset=+4] - -include::./edr-install-config/endpoint-protection-intro.asciidoc[leveloffset=+2] -include::./edr-install-config/deploy-endpoint-reqs.asciidoc[leveloffset=+3] -include::./edr-install-config/install-elastic-defend.asciidoc[leveloffset=+3] -include::./edr-install-config/deploy-endpoint-macos-cat-mont.asciidoc[leveloffset=+4] -include::./edr-install-config/deploy-endpoint-macos-ven.asciidoc[leveloffset=+4] -include::./edr-install-config/deploy-with-mdm.asciidoc[leveloffset=+4] -include::./edr-install-config/agent-tamper-protection.asciidoc[leveloffset=+4] -include::./edr-install-config/defend-feature-privs.asciidoc[leveloffset=+3] -include::./edr-install-config/configure-endpoint-integration-policy.asciidoc[leveloffset=+3] -include::./edr-install-config/artifact-control.asciidoc[leveloffset=+4] -include::./edr-install-config/endpoint-diagnostic-data.asciidoc[leveloffset=+4] -include::./edr-install-config/self-healing-rollback.asciidoc[leveloffset=+4] -include::./edr-install-config/linux-file-monitoring.asciidoc[leveloffset=+4] -include::./edr-install-config/endpoint-data-volume.asciidoc[leveloffset=+4] -include::./edr-install-config/uninstall-agent.asciidoc[leveloffset=+3] - -include::./edr-manage/manage-endpoint-protection.asciidoc[leveloffset=+2] -include::./edr-manage/endpoints-page.asciidoc[leveloffset=+3] -include::./edr-manage/policies-page-ov.asciidoc[leveloffset=+3] -include::./edr-manage/trusted-apps-ov.asciidoc[leveloffset=+3] -include::./edr-manage/event-filters.asciidoc[leveloffset=+3] -include::./edr-manage/host-isolation-exceptions.asciidoc[leveloffset=+3] -include::./edr-manage/blocklist.asciidoc[leveloffset=+3] -include::./edr-manage/optimize-edr.asciidoc[leveloffset=+3] -include::./edr-manage/endpoint-event-capture.asciidoc[leveloffset=+3] -include::./edr-manage/endpoint-protection-rules.asciidoc[leveloffset=+3] -include::./edr-manage/allowlist-endpoint-3rd-party-av.asciidoc[leveloffset=+3] -include::./edr-manage/endpoint-self-protection.asciidoc[leveloffset=+3] -include::./edr-manage/endpoint-command-ref.asciidoc[leveloffset=+3] - -include::./endpoint-response-actions/response-actions.asciidoc[leveloffset=+2] -include::./endpoint-response-actions/automated-response-actions.asciidoc[leveloffset=+3] -include::./endpoint-response-actions/host-isolation-ov.asciidoc[leveloffset=+3] -include::./endpoint-response-actions/response-actions-history.asciidoc[leveloffset=+3] -include::./endpoint-response-actions/third-party-actions.asciidoc[leveloffset=+3] -include::./endpoint-response-actions/response-actions-config.asciidoc[leveloffset=+3] - -include::./cloud-native-security/cloud-native-security-overview.asciidoc[leveloffset=+2] -include::./cloud-native-security/security-posture-management.asciidoc[leveloffset=+3] -include::./cloud-native-security/enable-cloudsec.asciidoc[leveloffset=+3] -include::./cloud-native-security/cspm.asciidoc[leveloffset=+3] -include::./cloud-native-security/cspm-get-started.asciidoc[leveloffset=+4] -include::./cloud-native-security/cspm-get-started-gcp.asciidoc[leveloffset=+4] -include::./cloud-native-security/cspm-get-started-azure.asciidoc[leveloffset=+4] -include::./cloud-native-security/cspm-permissions.asciidoc[leveloffset=+4] -include::./cloud-native-security/cspm-findings-page.asciidoc[leveloffset=+4] -include::./cloud-native-security/benchmark-rules.asciidoc[leveloffset=+4] -include::./cloud-native-security/cspm-cloud-posture-dashboard-dash.asciidoc[leveloffset=+4] -include::./cloud-native-security/cspm-security-posture-faq.asciidoc[leveloffset=+4] -include::./cloud-native-security/kspm.asciidoc[leveloffset=+3] -include::./cloud-native-security/get-started-with-kspm.asciidoc[leveloffset=+4] -include::./cloud-native-security/kspm-cspm-findings-page.asciidoc[leveloffset=+4] -include::./cloud-native-security/kspm-benchmark-rules.asciidoc[leveloffset=+4] -include::./cloud-native-security/kspm-cloud-posture-dashboard-dash.asciidoc[leveloffset=+4] -include::./cloud-native-security/security-posture-faq.asciidoc[leveloffset=+4] -include::./cloud-native-security/vuln-management-overview.asciidoc[leveloffset=+3] -include::./cloud-native-security/vuln-management-get-started.asciidoc[leveloffset=+4] -include::./cloud-native-security/vuln-management-findings.asciidoc[leveloffset=+4] -include::./cloud-native-security/vuln-management-dashboard-dash.asciidoc[leveloffset=+4] -include::./cloud-native-security/vuln-management-faq.asciidoc[leveloffset=+4] -include::./cloud-native-security/cloud-workload-protection.asciidoc[leveloffset=+3] -include::./cloud-native-security/environment-variable-capture.asciidoc[leveloffset=+4] -include::./cloud-native-security/ingest-cncf-data.asciidoc[leveloffset=+3] -include::./cloud-native-security/falco-setup.asciidoc[leveloffset=+4] -include::./cloud-native-security/aws-securityhub.asciidoc[leveloffset=+4] -include::./cloud-native-security/wiz.asciidoc[leveloffset=+4] - -include::./explore/explore-your-data.asciidoc[leveloffset=+2] -include::./explore/hosts-overview.asciidoc[leveloffset=+3] -include::./explore/network-page-overview.asciidoc[leveloffset=+3] -include::./explore/conf-map-ui.asciidoc[leveloffset=+4] -include::./explore/users-page.asciidoc[leveloffset=+3] -include::./explore/data-views-in-sec.asciidoc[leveloffset=+3] -include::./explore/runtime-fields.asciidoc[leveloffset=+3] -include::./explore/siem-field-reference.asciidoc[leveloffset=+3] - -include::./dashboards/dashboards-overview.asciidoc[leveloffset=+2] -include::./dashboards/overview-dashboard.asciidoc[leveloffset=+3] -include::./dashboards/detection-response-dashboard.asciidoc[leveloffset=+3] -include::./dashboards/cloud-posture-dashboard-dash.asciidoc[leveloffset=+3] -include::./dashboards/detection-entity-dashboard.asciidoc[leveloffset=+3] -include::./dashboards/data-quality-dash.asciidoc[leveloffset=+3] -include::./dashboards/vuln-management-dashboard-dash.asciidoc[leveloffset=+3] -include::./dashboards/rule-monitoring-dashboard.asciidoc[leveloffset=+3] - -include::./rules/detection-engine-overview.asciidoc[leveloffset=+2] -include::./rules/detections-permissions-section.asciidoc[leveloffset=+3] -include::./rules/detections-logsdb-impact.asciidoc[leveloffset=+3] - -include::./rules/about-rules.asciidoc[leveloffset=+2] -include::./rules/rules-ui-create.asciidoc[leveloffset=+3] -include::./rules/interactive-investigation-guides.asciidoc[leveloffset=+4] -include::./rules/building-block-rule.asciidoc[leveloffset=+4] -include::./rules/prebuilt-rules/prebuilt-rules-management.asciidoc[leveloffset=+3] -include::./rules/rules-ui-management.asciidoc[leveloffset=+3] -include::./rules/alerts-ui-monitor.asciidoc[leveloffset=+3] -include::./rules/detections-ui-exceptions.asciidoc[leveloffset=+3] -include::./rules/value-lists-exceptions.asciidoc[leveloffset=+4] -include::./rules/add-exceptions.asciidoc[leveloffset=+4] -include::./rules/shared-exception-lists.asciidoc[leveloffset=+4] -include::./rules/rules-coverage.asciidoc[leveloffset=+3] -include::./rules/tuning-detection-signals.asciidoc[leveloffset=+3] -include::./rules/prebuilt-rules/prebuilt-rules.asciidoc[leveloffset=+3] - -include::./alerts/alerts-ui-manage.asciidoc[leveloffset=+2] -include::./alerts/visualize-alerts.asciidoc[leveloffset=+3] -include::./alerts/view-alert-details.asciidoc[leveloffset=+3] -include::./alerts/signals-to-cases.asciidoc[leveloffset=+3] -include::./alerts/alert-suppression.asciidoc[leveloffset=+3] -include::./alerts/reduce-notifications-alerts.asciidoc[leveloffset=+3] -include::./alerts/query-alert-indices.asciidoc[leveloffset=+3] -include::./alerts/alert-schema.asciidoc[leveloffset=+3] - -include::./advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc[leveloffset=+2] -include::./advanced-entity-analytics/entity-risk-scoring.asciidoc[leveloffset=+3] -include::./advanced-entity-analytics/ers-req.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/asset-criticality.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/turn-on-risk-engine.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/analyze-risk-score-data.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/advanced-behavioral-detections.asciidoc[leveloffset=+3] -include::./advanced-entity-analytics/ml-requirements.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/machine-learning.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/tuning-anomaly-results.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/behavioral-detection-use-cases.asciidoc[leveloffset=+4] -include::./advanced-entity-analytics/prebuilt-ml-jobs.asciidoc[leveloffset=+4] - -include::./investigate/investigate-events.asciidoc[leveloffset=+2] -include::./investigate/timelines-ui.asciidoc[leveloffset=+3] -include::./investigate/timeline-templates-ui.asciidoc[leveloffset=+4] -include::./investigate/timeline-object-schema.asciidoc[leveloffset=+4] -include::./alerts/visual-event-analyzer.asciidoc[leveloffset=+3] -include::./cloud-native-security/session-view.asciidoc[leveloffset=+3] -include::./osquery/use-osquery.asciidoc[leveloffset=+3] -include::./osquery/osquery-response-action.asciidoc[leveloffset=+4] -include::./osquery/invest-guide-run-osquery.asciidoc[leveloffset=+4] -include::./osquery/alerts-run-osquery.asciidoc[leveloffset=+4] -include::./osquery/view-osquery-results.asciidoc[leveloffset=+4] -include::./osquery/osquery-placeholder-fields.asciidoc[leveloffset=+4] -include::./investigate/add-manage-notes.asciidoc[leveloffset=+3] -include::./investigate/indicators-of-compromise.asciidoc[leveloffset=+3] -include::./investigate/cases-overview.asciidoc[leveloffset=+3] -include::./investigate/case-permissions.asciidoc[leveloffset=+4] -include::./investigate/cases-open-manage.asciidoc[leveloffset=+4] -include::./investigate/cases-settings.asciidoc[leveloffset=+4] - -include::./assets/asset-management.asciidoc[leveloffset=+2] - -include::./settings/manage-settings.asciidoc[leveloffset=+2] -include::./settings/project-settings.asciidoc[leveloffset=+3] -include::./settings/advanced-settings.asciidoc[leveloffset=+3] - -include::./troubleshooting/troubleshooting-intro.asciidoc[leveloffset=+2] -include::./troubleshooting/ts-detection-rules.asciidoc[leveloffset=+3] -include::./troubleshooting/troubleshoot-endpoints.asciidoc[leveloffset=+3] \ No newline at end of file diff --git a/docs/serverless/rules/add-exceptions.asciidoc b/docs/serverless/rules/add-exceptions.asciidoc deleted file mode 100644 index 5013dc4db8..0000000000 --- a/docs/serverless/rules/add-exceptions.asciidoc +++ /dev/null @@ -1,285 +0,0 @@ -[[security-add-exceptions]] -= Add and manage exceptions - -// :description: Learn how to create and manage rule exceptions. -// :keywords: serverless, security, how-to, configure - - -You can add exceptions to a rule from the rule details page, the Alerts table, the alert details flyout, or the Shared Exception Lists page. When you add an exception, you can also close all alerts that meet the exception’s criteria. - -[IMPORTANT] -==== -* To ensure an exception is successfully applied, ensure that the fields you've defined for its query are correctly and consistently mapped in their respective indices. Refer to {ecs-ref}[ECS] to learn more about supported mappings. -* Be careful when adding exceptions to <> rules. Exceptions are evaluated against every event in the sequence, and if an exception matches any events that are necessary to complete the sequence, alerts are not created. -+ -To exclude values from a -specific event in the sequence, update the rule's EQL statement. For example: -+ -[source,eql] ----- -`sequence - [file where file.extension == "exe" - and file.name != "app-name.exe"] - [process where true - and process.name != "process-name.exe"]` ----- -* Be careful when adding exceptions to <> rules. Exceptions are evaluated against source and indicator indices, so if the exception matches events in _either_ index, alerts are not generated. -==== - -[discrete] -[[detection-rule-exceptions]] -== Add exceptions to a rule - -. Do one of the following: -+ -** To add an exception from the rule details page: -+ -... Go to the rule details page of the rule to which you want to add an -exception (**Rules** → **Detection rules (SIEM)** → **_Rule name_**). -... Scroll down the rule details page, select the **Rule exceptions** tab, then click **Add rule exception**. -+ -[role="screenshot"] -image::images/add-exceptions/-detections-rule-exception-tab.png[Detail of rule exceptions tab] -+ -** To add an exception from the Alerts table: -+ -... Go to **Alerts**. -... Scroll down to the Alerts table, go to the alert you want to create an exception for, click the **More Actions** menu (image:images/icons/boxesHorizontal.svg[Actions menu]), then select **Add rule exception**. -** To add an exception from the alert details flyout: -+ -... Go to **Alerts**. -... Click the **View details** button from the Alerts table. -... In the alert details flyout, click **Take action → Add rule exception**. -** To add an exception from the Shared Exception Lists page: -+ -... Go to **Rules** → **Shared exception lists**. -... Click **Create shared exception list** → **Create exception item**. -. In the **Add rule exception** flyout, name the exception. -. Add conditions that define the exception. When the exception's query evaluates to `true`, rules don't generate alerts even when their criteria are met. -+ -[IMPORTANT] -==== -Rule exceptions are case-sensitive, which means that any character that's entered as an uppercase or lowercase letter will be treated as such. In the event you _don't_ want a field evaluated as case-sensitive, some ECS fields have a `.caseless` version that you can use. -==== -+ -[NOTE] -==== -When you create a new exception from an alert, exception conditions are auto-populated with relevant alert data. Data from custom highlighted fields is listed first. A comment that describes the auto-generated exception conditions is also added to the **Add comments** section. -==== -+ -.. **Field**: Select a field to identify the event being filtered. -+ -[NOTE] -==== -A warning displays for fields with conflicts. Using these fields might cause unexpected exceptions behavior. Refer to <> for more information. -==== -.. **Operator**: Select an operator to define the condition: -+ -*** `is` | `is not` — Must be an exact match of the defined value. -*** `is one of` | `is not one of` — Matches any of the defined values. -*** `exists` | `does not exist` — The field exists. -*** `is in list` | `is not in list` — Matches values in a value list. -+ -[NOTE] -==== -* An exception defined by a value list must use `is in list` or `is not in list` in all conditions. -* Wildcards are not supported in value lists. -* If a value list can't be used due to <>, it'll be unavailable in the **Value** menu. -==== -*** `matches` | `does not match` — Allows you to use wildcards in **Value**, such as `C:\path\*\app.exe`. Available wildcards are `?` (match one character) and `*` (match zero or more characters). The selected **Field** data type must be {ref}/keyword.html#keyword-field-type[keyword], {ref}/text.html#text-field-type[text], or {ref}/keyword.html#wildcard-field-type[wildcard]. -+ -[NOTE] -==== -Some characters must be escaped with a backslash, such as `\` for a literal backslash, `*` for an asterisk, and `\?` for a question mark. Windows paths must be divided with double backslashes (for example, `C:\Windows\explorer.exe`), and paths that already include double backslashes might require four backslashes for each divider. -==== -+ -[IMPORTANT] -==== -Using wildcards can impact performance. To create a more efficient exception using wildcards, use multiple conditions and make them as specific as possible. For example, adding conditions using `process.name` or `file.name` can help limit the scope of wildcard matching. -==== -.. **Value**: Enter the value associated with the **Field**. To enter multiple values (when using `is one of` or `is not one of`), enter each value, then press **Return**. -+ -[NOTE] -==== -Identical, case-sensitive values are supported for the `is one of` and `is not one of` operators. For example, if you want to match the values `Windows` and `windows`, add both values to the **Value** field. -==== -+ -In the following example, the exception was created from the Rules page and prevents the rule from generating alerts when the `svchost.exe` process runs on hostname `siem-kibana`. -+ -[role="screenshot"] -image::images/add-exceptions/-detections-add-exception-ui.png[] -. Click **AND** or **OR** to create multiple conditions and define their relationships. -. Click **Add nested condition** to create conditions using nested fields. This is only required for -<>. For all other fields, nested conditions should not be used. -. Choose to add the exception to a rule or a shared exception list. -+ -[NOTE] -==== -If you are creating an exception from the Shared Exception Lists page, you can add the exception to multiple rules. -==== -+ -[TIP] -==== -If a shared exception list doesn't exist, you can <> from the Shared Exception Lists page. -==== -. (Optional) Enter a comment describing the exception. -. (Optional) Enter a future expiration date and time for the exception. -. Select one of the following alert actions: -+ -** **Close this alert**: Closes the alert when the exception is added. This option -is only available when adding exceptions from the Alerts table. -** **Close all alerts that match this exception and were generated by this rule**: Closes all alerts that match the exception's conditions and were generated only by the current rule. -. Click **Add rule exception**. - -[discrete] -[[endpoint-rule-exceptions]] -== Add {elastic-endpoint} exceptions - -You can add {elastic-endpoint} exceptions to <> or to rules that are associated with {elastic-endpoint} rule exceptions. To associate rules when creating or editing a rule, select the <> option. - -Endpoint exceptions are added to the endpoint protection rules **and** the {elastic-endpoint} on your hosts. - -[IMPORTANT] -==== -Exceptions added to the endpoint protection rules affect all alerts sent -from {elastic-endpoint}. Be careful not to unintentionally prevent useful Endpoint alerts. - -Additionally, to add an Endpoint exception to an endpoint protection rule, there must be at least one {elastic-endpoint} alert generated in the system. For non-production use, if no alerts exist, you can trigger a test alert using malware emulation techniques or tools such as the Anti Malware Testfile from the https://www.eicar.org/[European Institute for Computer Anti-Virus Research (EICAR)]. -==== - -[IMPORTANT] -==== -{ref}/binary.html[Binary fields] are not supported in detection rule exceptions. -==== - -. Do one of the following: -+ -** To add an Endpoint exception from the rule details page: -+ -... Go to the rule details page (**Rules** → **Detection rules (SIEM)**), and then search for and select one of the <>. -... Scroll down the rule details page, select the **Endpoint exceptions** tab, then click **Add endpoint exception**. -** To add an Endpoint exception from the Alerts table: -+ -... Go to **Alerts**. -... Scroll down to the Alerts table, and from an {elastic-endpoint} -alert, click the **More actions** menu (image:images/icons/boxesHorizontal.svg[Actions menu]), then select **Add Endpoint exception**. -** To add an Endpoint exception from Shared Exception Lists page: -+ -... Go to **Rules** → **Shared exception lists**. -... Expand the Endpoint Security Exception List or click the list name to open the list's details page. Next, click **Add endpoint exception**. -+ -[NOTE] -==== -The Endpoint Security Exception List is automatically created. By default, it's associated with endpoint protection rules and any rules with the <> option selected. -==== -+ -The **Add Endpoint Exception** flyout opens. -+ -[role="screenshot"] -image::images/add-exceptions/-detections-endpoint-add-exp.png[] -. If required, modify the conditions. Refer to <> for more information on when nested conditions are required. -+ -[IMPORTANT] -==== -Rule exceptions are case-sensitive, which means that any character that's entered as an uppercase or lowercase letter will be treated as such. In the event you _don't_ want a field evaluated as case-sensitive, some ECS fields have a `.caseless` version that you can use. -==== -+ -[NOTE] -==== -* Fields with conflicts are marked with a warning icon (image:images/icons/warning.svg[Warning]). Using these fields might cause unexpected exceptions behavior. For more information, refer to <>. -* Identical, case-sensitive values are supported for the `is one of` and `is not one of` operators. For example, if you want to match the values `Windows` and `windows`, add both values to the **Value** field. -==== -. (Optional) Add a comment to the exception. -. You can select any of the following: -+ -** **Close this alert**: Closes the alert when the exception is added. This option -is only available when adding exceptions from the Alerts table. -** **Close all alerts that match this exception and were generated by this rule**: -Closes all alerts that match the exception's conditions. -. Click **Add Endpoint Exception**. An exception is created for both the detection rule and the {elastic-endpoint}. -+ -[NOTE] -==== -It might take longer for exceptions to be applied to hosts within larger deployments. -==== - -[discrete] -[[ex-nested-conditions]] -== Exceptions with nested conditions - -Some Endpoint objects contain nested fields, and the only way to ensure you are -excluding the correct fields is with nested conditions. One example is the -`process.Ext` object: - -[source,json] ----- -{ - "ancestry": [], - "code_signature": { - "trusted": true, - "subject_name": "LFC", - "exists": true, - "status": "trusted" - }, - "user": "WDAGUtilityAccount", - "token": { - "elevation": true, - "integrity_level_name": "high", - "domain": "27FB305D-3838-4", - "user": "WDAGUtilityAccount", - "elevation_type": "default", - "sid": "S-1-5-21-2047949552-857980807-821054962-504" - } -} ----- - -Only these objects require nested conditions to ensure the exception functions -correctly: - -* `Endpoint.policy.applied.artifacts.global.identifiers` -* `Endpoint.policy.applied.artifacts.user.identifiers` -* `Target.dll.Ext.code_signature` -* `Target.process.Ext.code_signature` -* `Target.process.Ext.token.privileges` -* `Target.process.parent.Ext.code_signature` -* `Target.process.thread.Ext.token.privileges` -* `dll.Ext.code_signature` -* `file.Ext.code_signature` -* `file.Ext.macro.errors` -* `file.Ext.macro.stream` -* `process.Ext.code_signature` -* `process.Ext.token.privileges` -* `process.parent.Ext.code_signature` -* `process.thread.Ext.token.privileges` - -[discrete] -[[security-add-exceptions-nested-condition-example]] -=== Nested condition example - -Creates an exception that excludes all LFC-signed trusted processes: - -[role="screenshot"] -image::images/add-exceptions/-detections-nested-exp.png[] - -[discrete] -[[manage-exception]] -== View and manage exceptions - -To view a rule's exceptions, open the rule's details page (**Rules** → **Detection rules (SIEM)** → **_Rule name_**), then scroll down and select the **Rule exceptions** or **Endpoint exceptions** tab. All exceptions that belong to the rule will display in a list. From the list, you can filter, edit, and delete exceptions. You can also toggle between **Active exceptions** and **Expired exceptions**. - -[role="screenshot"] -image::images/add-exceptions/-detections-manage-default-rule-list.png[A default rule list] - -[discrete] -[[rules-using-same-exception]] -== Find rules using the same exceptions - -To find out if an exception is used by other rules, select the **Rule exceptions** or **Endpoint exceptions** tab, navigate to an exception list item, then click **Affects _X_ rules**. - -[NOTE] -==== -Changes that you make to the exception also apply to other rules that use the exception. -==== - -[role="screenshot"] -image::images/add-exceptions/-detections-exception-affects-multiple-rules.png[Exception that affects multiple rules] diff --git a/docs/serverless/rules/detection-engine-overview.asciidoc b/docs/serverless/rules/detection-engine-overview.asciidoc deleted file mode 100644 index 3d89447e5c..0000000000 --- a/docs/serverless/rules/detection-engine-overview.asciidoc +++ /dev/null @@ -1,134 +0,0 @@ -[[security-detection-engine-overview]] -= Detection engine overview - -// :description: Learn about the detection engine and its features. -// :keywords: serverless, security, overview - - -Use the detection engine to create and manage rules and view the alerts -these rules create. Rules periodically search indices (such as `logs-*` and -`filebeat-*`) for suspicious source events and create alerts when a rule's -conditions are met. When an alert is created, its status is `Open`. To help -track investigations, an alert's <> can be set as -`Open`, `Acknowledged`, or `Closed`. - -[role="screenshot"] -image::images/detection-engine-overview/-detections-alert-page.png[Alerts page] - -In addition to creating <>, enable -<> to immediately start detecting -suspicious activity. For detailed information on all the prebuilt rules, see the <>. Once the prebuilt rules are loaded and -running, <> and <> explain -how to modify the rules to reduce false positives and get a better set of -actionable alerts. You can also use exceptions and value lists when creating or -modifying your own rules. - -There are several special prebuilt rules you need to know about: - -// Links to prebuilt rule pages temporarily removed for initial serverless docs. - -* <>: Automatically create alerts based on {elastic-defend}'s threat monitoring and prevention. - -// Links to prebuilt rule pages temporarily removed for initial serverless docs. - -* **External Alerts**: Automatically creates an alert for -all incoming third-party system alerts (for example, Suricata alerts). - -If you want to receive notifications via external systems, such as Slack or -email, when alerts are created, use the {kibana-ref}/alerting-getting-started.html[Alerting and Actions] framework. - -After rules have started running, you can monitor their executions to verify -they are functioning correctly, as well as view, manage, and troubleshoot -alerts (see <> and <>). - -You can create and manage rules and alerts via the UI or the {security-guide}/rule-api-overview.html[Detections API]. - -// Link to classic docs until serverless API docs are available. - -[IMPORTANT] -==== -To make sure you can access Detections and manage rules, see -<>. -==== - -[discrete] -[[support-indicator-rules]] -== Limited support for indicator match rules - -Indicator match rules provide a powerful capability to search your security data; however, their queries can consume significant deployment resources. When creating an <>, we recommend limiting the time range of the indicator index query to the minimum period necessary for the desired rule coverage. For example, the default indicator index query `@timestamp > "now-30d/d"` searches specified indicator indices for indicators ingested during the past 30 days and rounds the query start time down to the nearest day (resolves to UTC `00:00:00`). Without this limitation, the rule will include all of the indicators in your indicator indices, which may extend the time it takes for the indicator index query to complete. - -In addition, indicator match rules with an additional look-back time value greater than 24 hours are not supported. - -[discrete] -[[detections-permissions]] -== Detections configuration and prerequisites - -<> provides detailed information on all the -permissions required to initiate and use the Detections feature. - -[discrete] -[[malware-prevention]] -== Malware prevention - -Malware, short for malicious software, is any software program designed to damage or execute unauthorized actions on a -computer system. Examples of malware include viruses, worms, Trojan horses, adware, scareware, and spyware. Some -malware, such as viruses, can severely damage a computer's hard drive by deleting files or directory information. Other -malware, such as spyware, can obtain user data without their knowledge. - -Malware may be stealthy and appear as legitimate executable code, scripts, active content, and other software. It is also -often embedded in non-malicious files, non-suspicious websites, and standard programs — sometimes making the root -source difficult to identify. If infected and not resolved promptly, malware can cause irreparable damage to a computer -network. - -For information on how to enable malware protection on your host, see <>. - -[discrete] -[[machine-learning-model]] -=== Machine learning model - -To determine if a file is malicious or benign, a machine learning model looks for static attributes of files (without executing -the file) that include file structure, layout, and content. This includes information such as file header data, imports, exports, -section names, and file size. These attributes are extracted from millions of benign and malicious file samples, which then -are passed to a machine-learning algorithm that distinguishes a benign file from a malicious one. The machine learning -model is updated as new data is procured and analyzed. - -[discrete] -[[security-detection-engine-overview-threshold]] -=== Threshold - -A malware threshold determines the action the agent should take if malware is detected. The Elastic Agent uses a recommended threshold level that generates a balanced number of alerts with a low probability of undetected malware. This threshold also minimizes the number of false positive alerts. - -[discrete] -[[ransomware-prevention]] -== Ransomware prevention - -Ransomware is computer malware that installs discreetly on a user's computer and encrypts data until a specified amount of money (ransom) is paid. Ransomware is usually similar to other malware in its delivery and execution, infecting systems -through spear-phishing or drive-by downloads. If not resolved immediately, ransomware can cause irreparable damage to an entire computer network. - -Behavioral ransomware prevention on the Elastic Endpoint detects and stops ransomware attacks on Windows systems by analyzing data from low-level system processes, and is effective across an array of widespread ransomware families — including those targeting the system’s master boot record. - -For information on how to enable ransomware protection on your host, see <>. - -[discrete] -[[security-detection-engine-overview-resolve-ui-error-messages]] -=== Resolve UI error messages - -Depending on your user role privileges and whether detection system indices have already been created, you might get one of these error messages when you -open the **Alerts** or **Rules** page: - -* **`Let’s set up your detection engine`** -+ -If you get this message, a user with specific privileges must visit the -**Alerts** or **Rules** page before you can view detection alerts and rules. -Refer to <> for a list of all the requirements. -* **`Detection engine permissions required`** -+ -If you get this message, you do not have the -<> to view the **Detections** feature, -and you should contact your project administrator. - -[discrete] -[[detections-logsdb-index-mode]] -== Using logsdb index mode - -logsdb is enabled by default for Elastic serverless. Refer to <> to learn more. \ No newline at end of file diff --git a/docs/serverless/rules/rules-ui-create.asciidoc b/docs/serverless/rules/rules-ui-create.asciidoc deleted file mode 100644 index 736ea49017..0000000000 --- a/docs/serverless/rules/rules-ui-create.asciidoc +++ /dev/null @@ -1,896 +0,0 @@ -[[security-rules-create]] -= Create a detection rule - -// :description: Create detection rules to monitor your environment for suspicious and malicious behavior. -// :keywords: serverless, security, defend, how-to, manage, secure - - -To create a new detection rule, follow these steps: - -. Define the <>. The configuration for this step varies depending on the rule type. -. Configure basic rule settings. -. Configure advanced rule settings (optional). -. Set the rule's schedule. -. Set up rule actions (optional). -. Set up response actions (optional). - -.Requirements -[NOTE] -==== -* To create detection rules, you must have access to data views, which requires the appropriate user role. -* You'll also need permissions to enable and view detections, manage rules, manage alerts, and preview rules. These permissions depend on the user role. Refer to <> for more information. -==== - -[TIP] -==== -At any step, you can <> before saving it to see what kind of results you can expect. -==== - -[discrete] -[[create-custom-rule]] -== Create a custom query rule - -. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. -. To create a rule based on a KQL or Lucene query, select **Custom query**, -then: -+ -.. Define which {es} indices or data view the rule searches for alerts. -.. Use the filter and query fields to create the criteria used for detecting -alerts. -+ -The following example (based on the prebuilt rule Volume Shadow Copy Deleted or Resized via VssAdmin) detects when the `vssadmin delete shadows` -Windows command is executed: -+ -*** **Index patterns**: `winlogbeat-*` -+ -Winlogbeat ships Windows event logs to {elastic-sec}. -*** **Custom query**: `event.action:"Process Create (rule: ProcessCreate)" and process.name:"vssadmin.exe" and process.args:("delete" and "shadows")` -+ -Searches the `winlogbeat-*` indices for `vssadmin.exe` executions with -the `delete` and `shadow` arguments, which are used to delete a volume's shadow -copies. -+ -[role="screenshot"] -image::images/rules-ui-create/-detections-rule-query-example.png[Rule query example] -.. You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. -+ -When you use a saved query, the **Load saved query "_query name_" dynamically on each rule execution** check box appears: -+ -*** Select this to use the saved query every time the rule runs. This links the rule to the saved query, and you won't be able to modify the rule's **Custom query** field or filters because the rule will only use settings from the saved query. To make changes, modify the saved query itself. -*** Deselect this to load the saved query as a one-time way of populating the rule's **Custom query** field and filters. This copies the settings from the saved query to the rule, so you can then further adjust the rule's query and filters as needed. If the saved query is later changed, the rule will not inherit those changes. -. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. -+ -//// -/* The following steps are repeated across multiple rule types. If you change anything -in these steps or sub-steps, apply the change to the other rule types, too. */ -//// -+ -. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. -+ -.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. -.. Enter the field's data type. -. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. -+ -.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. -.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. -. Click **Continue** to <>. - -[discrete] -[[create-ml-rule]] -== Create a machine learning rule - -[IMPORTANT] -==== -To create or edit {ml} rules, you need an appropriate user role. Additionally, the selected {ml} job must be running for the rule to function correctly. -==== - -. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. -. To create a rule based on a {ml} anomaly threshold, select **Machine Learning**, -then select: -+ -.. The required {ml} jobs. -+ -[NOTE] -==== -If a required job isn't currently running, it will automatically start when you finish configuring and enable the rule. -==== -.. The anomaly score threshold above which alerts are created. -. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. -+ -[NOTE] -==== -Because {ml} rules generate alerts from anomalies, which don't contain source event fields, you can only use anomaly fields when configuring alert suppression. -==== -+ -//// -/* The following steps are repeated across multiple rule types. If you change anything -in these steps or sub-steps, apply the change to the other rule types, too. */ -//// -. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. -+ -.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. -.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. -. Click **Continue** to <>. - -[discrete] -[[create-threshold-rule]] -== Create a threshold rule - -. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. -. To create a rule based on a source event field threshold, select **Threshold**, then: -+ -.. Define which {es} indices the rule analyzes for alerts. -.. Use the filter and query fields to create the criteria used for detecting -alerts. -+ -[NOTE] -==== -You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. -==== -.. Use the **Group by** and **Threshold** fields to determine which source event field is used as a threshold and the threshold's value. -.. Use the **Count** field to limit alerts by cardinality of a certain field. -+ -For example, if **Group by** is `source.ip, destination.ip` and its **Threshold** is `10`, an alert is generated for every pair of source and destination IP addresses that appear in at least 10 of the rule's search results. -+ -You can also leave the **Group by** field undefined. The rule then creates an alert when the number of search results is equal to or greater than the threshold value. If you set **Count** to limit the results by `process.name` >= 2, an alert will only be generated for source/destination IP pairs that appear with at least 2 unique process names across all events. -+ -[IMPORTANT] -==== -Alerts created by threshold rules are synthetic alerts that do not resemble the source documents. The alert itself only contains data about the fields that were aggregated over (the **Group by** fields). Other fields are omitted, because they can vary across all source documents that were counted toward the threshold. Additionally, you can reference the actual count of documents that exceeded the threshold from the `kibana.alert.threshold_result.count` field. -==== -. (Optional) Select **Suppress alerts** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. -+ -//// -/* The following steps are repeated across multiple rule types. If you change anything -in these steps or sub-steps, apply the change to the other rule types, too. */ -//// -. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. -+ -.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. -.. Enter the field's data type. -. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. -+ -.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. -.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. -. Click **Continue** to <>. - -[discrete] -[[create-eql-rule]] -== Create an event correlation rule - -. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. -. To create an event correlation rule using EQL, select **Event Correlation**, then: -+ -.. Define which {es} indices or data view the rule searches when querying for events. -.. Write an {ref}/eql-syntax.html[EQL query] that searches for matching events or a series of matching events. -+ -[TIP] -==== -To find events that are missing in a sequence, use the {ref}/eql-syntax.html#eql-missing-events[missing events] syntax. -==== -+ -For example, the following rule detects when `msxsl.exe` makes an outbound -network connection: -+ -*** **Index patterns**: `winlogbeat-*` -+ -Winlogbeat ships Windows events to {elastic-sec}. -*** **EQL query**: -+ -[source,eql] ----- -sequence by process.entity_id - [process - where event.type in ("start", "process_started") - and process.name == "msxsl.exe"] - [network - where event.type == "connection" - and process.name == "msxsl.exe" - and network.direction == "outgoing"] ----- -+ -Searches the `winlogbeat-*` indices for sequences of a `msxsl.exe` process start -event followed by an outbound network connection event that was started by the -`msxsl.exe` process. -+ -[role="screenshot"] -image::images/rules-ui-create/-detections-eql-rule-query-example.png[] -+ -[NOTE] -==== -For sequence events, the {security-app} generates a single alert when all events listed in the sequence are detected. To see the matched sequence events in more detail, you can view the alert in the Timeline, and, if all events came from the same process, open the alert in Analyze Event view. -==== -. (Optional) Click the EQL settings icon (image:images/icons/controlsVertical.svg[EQL settings]) to configure additional fields used by {ref}/eql.html#specify-a-timestamp-or-event-category-field[EQL search]: -+ -** **Event category field**: Contains the event classification, such as `process`, `file`, or `network`. This field is typically mapped as a field type in the {ref}/keyword.html[keyword family]. Defaults to the `event.category` ECS field. -** **Tiebreaker field**: Sets a secondary field for sorting events (in ascending, lexicographic order) if they have the same timestamp. -** **Timestamp field**: Contains the event timestamp used for sorting a sequence of events. This is different from the **Timestamp override** advanced setting, which is used for querying events within a range. Defaults to the `@timestamp` ECS field. -. preview:[] (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. -+ -//// -/* The following steps are repeated across multiple rule types. If you change anything -in these steps or sub-steps, apply the change to the other rule types, too. */ -//// -. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. -+ -.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. -.. Enter the field's data type. -. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. -+ -.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. -.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. -. Click **Continue** to <>. - -[discrete] -[[create-indicator-rule]] -== Create an indicator match rule - -[NOTE] -==== -{elastic-sec} provides limited support for indicator match rules. See <> for more information. -==== - -. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. -. To create a rule that searches for events whose specified field value matches the specified indicator field value in the indicator index patterns, select **Indicator Match**, then fill in the following fields: -+ -.. **Source**: The individual index patterns or data view that specifies what data to search. -.. **Custom query**: The query and filters used to retrieve the required results from -the {elastic-sec} event indices. For example, if you want to match documents that only contain a `destination.ip` address field, add `destination.ip : *`. -+ -[TIP] -==== -If you want the rule to check every field in the indices, use this -wildcard expression: `*:*`. -==== -+ -[NOTE] -==== -You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. -==== -.. **Indicator index patterns**: The indicator index patterns containing field values for which you want to generate alerts. This field is automatically populated with indices specified in the `securitySolution:defaultThreatIndex` advanced setting. For more information, see <>. -+ -[IMPORTANT] -==== -Data in indicator indices must be <>, and so it must contain a `@timestamp` field. -==== -.. **Indicator index query**: The query and filters used to filter the fields from -the indicator index patterns. The default query `@timestamp > "now-30d/d"` searches specified indicator indices for indicators ingested during the past 30 days and rounds the start time down to the nearest day (resolves to UTC `00:00:00`). -.. **Indicator mapping**: Compares the values of the specified event and indicator fields, and generates an alert if the values are identical. -+ -[NOTE] -==== -Only single-value fields are supported. -==== -+ -To define which field values are compared from the indices, add the following: -+ -*** **Field**: The field used for comparing values in the {elastic-sec} event -indices. -*** **Indicator index field**: The field used for comparing values in the indicator -indices. -.. You can add `AND` and `OR` clauses to define when alerts are generated. -+ -For example, to create a rule that generates alerts when `host.name` **and** -`destination.ip` field values in the `logs-*` or `packetbeat-*` {elastic-sec} indices -are identical to the corresponding field values in the `mock-threat-list` indicator -index, enter the rule parameters seen in the following image: -+ -[role="screenshot"] -image::images/rules-ui-create/-detections-indicator-rule-example.png[Indicator match rule settings] -+ -[TIP] -==== -Before you create rules, create <> so you can select them under **Timeline template** at the end of the **Define rule** section. When alerts generated by the rule are investigated in the Timeline, Timeline query values are replaced with their corresponding alert field values. -==== -. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. -+ -//// -/* The following steps are repeated across multiple rule types. If you change anything -in these steps or sub-steps, apply the change to the other rule types, too. */ -//// -. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. -+ -.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. -.. Enter the field's data type. -. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. -+ -.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. -.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. -. Click **Continue** to <>. - -[discrete] -[[indicator-value-lists]] -=== Use value lists with indicator match rules - -While there are numerous ways you can add data into indicator indices, you can use value lists as the indicator match index in an indicator match rule. Take the following scenario, for example: - -You uploaded a value list of known ransomware domains, and you want to be notified if any of those domains matches a value contained in a domain field in your security event index pattern. - -. Upload a value list of indicators. -. Create an indicator match rule and fill in the following fields: -+ -.. **Index patterns**: The Elastic Security event indices on which the rule runs. -.. **Custom query**: The query and filters used to retrieve the required results from the Elastic Security event indices (e.g., `host.domain :*`). -.. **Indicator index patterns**: Value lists are stored in a hidden index called `.items-`. Enter the name of the {kib} space in which this rule will run in this field. -.. **Indicator index query**: Enter the value `list_id :`, followed by the name of the value list you want to use as your indicator index (uploaded in Step 1 above). -.. **Indicator mapping** -+ -*** **Field**: Enter the field from the Elastic Security event indices to be used for comparing values. -*** **Indicator index field**: Enter the type of value list you created (i.e., `keyword`, `text`, or `IP`). -+ -[TIP] -==== -If you don't remember this information, go to **Rules** → **Detection rules (SIEM)** → **Manage value lists**. Locate the appropriate value list and note the field in the corresponding `Type` column. (Examples include keyword, text, and IP.) -==== - -[role="screenshot"] -image::images/rules-ui-create/-detections-indicator_value_list.png[] - -[discrete] -[[create-new-terms-rule]] -== Create a new terms rule - -. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page displays. -. To create a rule that searches for each new term detected in source documents, select **New Terms**, then: -+ -.. Specify what data to search by entering individual {es} index patterns or selecting an existing data view. -.. Use the filter and query fields to create the criteria used for detecting -alerts. -+ -[NOTE] -==== -You can use saved queries and queries from saved Timelines (**Import query from saved Timeline**) as rule conditions. -==== -.. Use the **Fields** menu to select a field to check for new terms. You can also select up to three fields to detect a combination of new terms (for example, a `host.ip` and `host.id` that have never been observed together before). -+ -[IMPORTANT] -==== -When checking multiple fields, each unique combination of values from those fields is evaluated separately. For example, a document with `host.name: ["host-1", "host-2", "host-3"]` and `user.name: ["user-1", "user-2", "user-3"]` has 9 (3x3) unique combinations of `host.name` and `user.name`. A document with 11 values in `host.name` and 10 values in `user.name` has 110 (11x10) unique combinations. The new terms rule only evaluates 100 unique combinations per document, so selecting fields with large arrays of values might cause incorrect results. -==== -.. Use the **History Window Size** menu to specify the time range to search in minutes, hours, or days to determine if a term is new. The history window size must be larger than the rule interval plus additional look-back time, because the rule will look for terms where the only time(s) the term appears within the history window is _also_ within the rule interval and additional look-back time. -+ -For example, if a rule has an interval of 5 minutes, no additional look-back time, and a history window size of 7 days, a term will be considered new only if the time it appears within the last 7 days is also within the last 5 minutes. Configure the rule interval and additional look-back time when you <>. -. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. -+ -//// -/* The following steps are repeated across multiple rule types. If you change anything -in these steps or sub-steps, apply the change to the other rule types, too. */ -//// -. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. -+ -.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. -.. Enter the field's data type. -. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. -+ -.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. -.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. -. Click **Continue** to <>. - -[discrete] -[[create-esql-rule]] -== Create an {esql} rule - -Use {ref}/esql.html[{esql}] to query your source events and aggregate event data. Query results are returned in a table with rows and columns. Each row becomes an alert. - -To create an {esql} rule: - -. Go to **Rules** → **Detection rules (SIEM)** → **Create new rule**. The **Create new rule** page appears. -. Select **{esql}**, then write a query. -+ -[NOTE] -==== -Refer to the sections below to learn more about <>, <>, and <>. -==== -+ -[TIP] -==== -Click the help icon (image:images/icons/iInCircle.svg[Click the ES|QL help icon]) to open the in-product reference documentation for all {esql} commands and functions. -==== -. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. -+ -//// -/* The following steps are repeated across multiple rule types. If you change anything -in these steps or sub-steps, apply the change to the other rule types, too. */ -//// -. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. -+ -.. Click **Add required field**, then select a field from the index patterns or data view you specified for the rule. You can also start typing a field's name to find it faster, or type in an entirely new custom field. -.. Enter the field's data type. -. (Optional) Add **Related integrations** to associate the rule with one or more {integrations-docs}[Elastic integrations]. This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's <> when viewing the rule. -+ -.. Click **Add integration**, then select an integration from the list. You can also start typing an integration's name to find it faster. -.. Enter the version of the integration you want to associate with the rule, using https://semver.org/[semantic versioning]. For version ranges, you must use tilde or caret syntax. For example, `~1.2.3` is from 1.2.3 to any patch version less than 1.3.0, and `^1.2.3` is from 1.2.3 to any minor and patch version less than 2.0.0. -. Click **Continue** to <>. - -[discrete] -[[esql-rule-query-types]] -=== {esql} query types - -{esql} rule queries are loosely categorized into two types: aggregating and non-aggregating. - -[discrete] -[[esql-agg-query]] -==== Aggregating query - -Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and any new fields that the query creates. - -[NOTE] -==== -A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the `STATS...BY` function to create a column with aggregated values, the column is created when the rule runs and is added as a new field to any alerts that are generated by the rule. -==== - -Here is an example aggregating query: - -[source,esql] ----- -FROM logs-* -| STATS host_count = COUNT(host.name) BY host.name -| SORT host_count DESC -| WHERE host_count > 20 ----- - -* This query starts by searching logs from indices that match the pattern `logs-*`. -* The query then aggregates the count of events by `host.name`. -* Next, it sorts the result by `host_count` in descending order. -* Then, it filters for events where the `host_count` field appears more than 20 times during the specified rule interval. - -[NOTE] -==== -Rules that use aggregating queries might create duplicate alerts. This can happen when events that occur in the additional look-back time are aggregated both in the current rule execution and in a previous rule execution. -==== - -[discrete] -[[esql-non-agg-query]] -==== Non-aggregating query - -Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. - -[NOTE] -==== -A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the columns are created when the rule runs and are added as new fields to any alerts generated by the rule. -==== - -Here is an example non-aggregating query: - -[source,esql] ----- -FROM logs-* METADATA _id, _index, _version -| WHERE event.category == "process" AND event.id == "8a4f500d" -| LIMIT 10 ----- - -* This query starts by querying logs from indices that match the pattern `logs-*`. The `METADATA _id, _index, _version` operator allows <>. -* Next, the query filters events where the `event.category` is a process and the `event.id` is `8a4f500d`. -* Then, it limits the output to the top 10 results. - -[discrete] -[[esql-non-agg-query-dedupe]] -==== Turn on alert deduplication for rules using non-aggregating queries - -To deduplicate alerts, a query needs access to the `_id`, `_index`, and `_version` metadata fields of the queried source event documents. You can allow this by adding the `METADATA _id, _index, _version` operator after the `FROM` source command, for example: - -[source,esql] ----- -FROM logs-* METADATA _id, _index, _version -| WHERE event.category == "process" AND event.id == "8a4f500d" -| LIMIT 10 ----- - -When those metadata fields are provided, unique alert IDs are created for each alert generated by the query. - -When developing the query, make sure you don't {ref}/esql-commands.html#esql-drop[`DROP`] or filter out the `_id`, `_index`, or `_version` metadata fields. - -Here is an example of a query that fails to deduplicate alerts. It uses the `DROP` command to omit the `_id` property from the results table: - -[source,esql] ----- -FROM logs-* METADATA _id, _index, _version -| WHERE event.category == "process" AND event.id == "8a4f500d" -| DROP _id -| LIMIT 10 ----- - -Here is another example of an invalid query that uses the `KEEP` command to only return `event.*` fields in the results table: - -[source,esql] ----- -FROM logs-* METADATA _id, _index, _version -| WHERE event.category == "process" AND event.id == "8a4f500d" -| KEEP event.* -| LIMIT 10 ----- - -[discrete] -[[esql-query-design]] -=== Query design considerations - -When writing your query, consider the following: - -* The {ref}/esql-commands.html#esql-limit[`LIMIT`] command specifies the maximum number of rows an {esql} query returns and the maximum number of alerts created per rule run. Similarly, a detection rule's **Max alerts per run** setting specifies the maximum number of alerts it can create every time it runs. -+ -If the `LIMIT` value and **Max alerts per run** value are different, the rule uses the lower value to determine the maximum number of alerts the rule generates. -* When writing an aggregating query, use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] command with fields that you want to search and filter for after alerts are created. For example, using the `host.name`, `user.name`, `process.name` fields with the `BY` operator of the `STATS...BY` command returns these fields in alert documents, and allows you to search and filter for them from the Alerts table. -* When configuring alert suppression on a non-aggregating query, we recommend sorting results by ascending `@timestamp` order. Doing so ensures that alerts are properly suppressed, especially if the number of alerts generated is higher than the **Max alerts per run** value. - -[discrete] -[[esql-rule-limitations]] -=== {esql} rule limitations - -If your {esql} query creates new fields that aren’t part of the ECS schema, they aren't mapped to the alerts index, so you can't search for or filter them in the Alerts table. As a workaround, create <>. - -[discrete] -[[custom-highlighted-esql-fields]] -=== Highlight fields returned by the {esql} rule query - -When configuring an {esql} rule's **<>**, you can specify any fields that the rule's aggregating or non-aggregating query return. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. - -[discrete] -[[rule-ui-basic-params]] -== Configure basic rule settings - -. In the **About rule** pane, fill in the following fields: -+ -.. **Name**: The rule's name. -.. **Description**: A description of what the rule does. -.. **Default severity**: Select the severity level of alerts created by the rule: -+ -*** **Low**: Alerts that are of interest but generally are not considered to be security incidents. Sometimes a combination of low severity alerts can indicate suspicious activity. -*** **Medium**: Alerts that require investigation. -*** **High**: Alerts that require an immediate investigation. -*** **Critical**: Alerts that indicate it is highly likely a security incident has occurred. -.. **Severity override** (optional): Select to use source event values to -override the **Default severity** in generated alerts. When selected, a UI -component is displayed where you can map the source event field values to -severity levels. The following example shows how to map severity levels to `host.name` -values: -+ -[role="screenshot"] -image::images/rules-ui-create/-detections-severity-mapping-ui.png[] -+ -[NOTE] -==== -For threshold rules, not all source event values can be used for overrides; only the fields that were aggregated over (the `Group by` fields) will contain data. Please also note that overrides are not supported for event correlation rules. -==== -.. **Default risk score**: A numerical value between 0 and 100 that indicates the risk of events detected by the rule. This setting changes to a default value when you change the **Severity** level, but you can adjust the risk score as needed. General guidelines are: -+ -*** `0` - `21` represents low severity. -*** `22` - `47` represents medium severity. -*** `48` - `73` represents high severity. -*** `74` - `100` represents critical severity. -.. **Risk score override** (optional): Select to use a source event value to -override the **Default risk score** in generated alerts. When selected, a UI -component is displayed to select the source field used for the risk -score. For example, if you want to use the source event's risk score in -alerts: -+ -[role="screenshot"] -image::images/rules-ui-create/-detections-risk-source-field-ui.png[] -+ -[NOTE] -==== -For threshold rules, not all source event values can be used for overrides; only the fields that were aggregated over (the `Group by` fields) will contain data. -==== -.. **Tags** (optional): Words and phrases used to categorize, filter, and search -the rule. -. Continue with **one** of the following: -+ -** <> -** <> - -[discrete] -[[rule-ui-advanced-params]] -== Configure advanced rule settings (optional) - -. Click **Advanced settings** and fill in the following fields where applicable: -+ -.. **Reference URLs** (optional): References to information that is relevant to -the rule. For example, links to background information. -.. **False positive examples** (optional): List of common scenarios that may produce -false-positive alerts. -.. **MITRE ATT&CK^TM^ threats** (optional): Add relevant https://attack.mitre.org/[MITRE] framework tactics, techniques, and subtechniques. -.. **Custom highlighted fields** (optional): Specify highlighted fields for unique alert investigation flows. You can choose any fields that are available in the you selected for the rule's data source. -+ -After you create the rule, you can find all custom highlighted fields in the About section of the rule details page. If the rule has alerts, you can find custom highlighted fields in the <> section of the alert details flyout. -.. **Setup guide** (optional): Instructions on rule prerequisites such as required integrations, configuration steps, and anything else needed for the rule to work correctly. -.. **Investigation guide** (optional): Information for analysts investigating -alerts created by the rule. You can also add action buttons to <> or <> using alert data. -.. **Author** (optional): The rule's authors. -.. **License** (optional): The rule's license. -.. **Elastic endpoint exceptions** (optional): Adds all <> to this rule. -+ -[NOTE] -==== -If you select this option, you can add {elastic-endpoint} exceptions on the Rule details page. Additionally, all future exceptions added to <> will also affect this rule. -==== -.. **Building block** (optional): Select to create a building-block rule. By default, alerts generated from a building-block rule are not displayed in the UI. See <> for more information. -.. **Max alerts per run** (optional): Specify the maximum number of alerts the rule can create each time it runs. Default is 100. -.. **Indicator prefix override**: Define the location of indicator data within the structure of indicator documents. When the indicator match rule executes, it queries specified indicator indices and references this setting to locate fields with indicator data. This data is used to enrich indicator match alerts with metadata about matched threat indicators. The default value for this setting is `threat.indicator`. -+ -[IMPORTANT] -==== -If your threat indicator data is at a different location, update this setting accordingly to ensure alert enrichment can still be performed. -==== -.. **Rule name override** (optional): Select a source event field to use as the -rule name in the UI (Alerts table). This is useful for exposing, at a glance, -more information about an alert. For example, if the rule generates alerts from -Suricata, selecting `event.action` lets you see what action (Suricata category) -caused the event directly in the Alerts table. -+ -[NOTE] -==== -For threshold rules, not all source event values can be used for overrides; only the fields that were aggregated over (the `Group by` fields) will contain data. -==== -.. **Timestamp override** (optional): Select a source event timestamp field. When selected, the rule's query uses the selected field, instead of the default `@timestamp` field, to search for alerts. This can help reduce missing alerts due to network or server outages. Specifically, if your ingest pipeline adds a timestamp when events are sent to {es}, this avoids missing alerts due to ingestion delays. -However, if you know your data source has an inaccurate `@timestamp` value, it is recommended you select the **Do not use @timestamp as a fallback timestamp field** option to ignore the `@timestamp` field entirely. -+ -[TIP] -==== -The {filebeat-ref}/filebeat-module-microsoft.html[Microsoft] and -{filebeat-ref}/filebeat-module-google_workspace.html[Google Workspace] {filebeat} modules have an `event.ingested` timestamp field that can be used instead of the default `@timestamp` field. -==== -. Click **Continue**. The **Schedule rule** pane is displayed. -+ -[role="screenshot"] -image::images/rules-ui-create/-detections-schedule-rule.png[] -. Continue with <>. - -[discrete] -[[rule-schedule]] -== Set the rule's schedule - -. Select how often the rule runs. -. Optionally, add `Additional look-back time` to the rule. When defined, the -rule searches indices with the additional time. -+ -For example, if you set a rule to run every 5 minutes with an additional -look-back time of 1 minute, the rule runs every 5 minutes but analyzes the -documents added to indices during the last 6 minutes. -+ -[IMPORTANT] -==== -It is recommended to set the `Additional look-back time` to at -least 1 minute. This ensures there are no missing alerts when a rule does not -run exactly at its scheduled time. - -{elastic-sec} prevents duplication. Any duplicate alerts that are discovered during the -`Additional look-back time` are _not_ created. -==== -. Click **Continue**. The **Rule actions** pane is displayed. -. Do either of the following: -+ -** Continue onto <> and <> (optional). -** Create the rule (with or without activation). - -[discrete] -[[rule-notifications]] -== Set up rule actions (optional) - -Use actions to set up notifications sent via other systems when alerts are generated. - -[NOTE] -==== -To use actions for alert notifications, you need the appropriate user role. For more information, see <>. -==== - -. Select a connector type to determine how notifications are sent. For example, if you select the {jira} connector, notifications are sent to your {jira} system. -+ -[NOTE] -==== -Each action type requires a connector. Connectors store the -information required to send the notification from the external system. You can -configure connectors while creating the rule or in **Project settings** → **Stack Management** → **{connectors-ui}**. For more -information, see {kibana-ref}/action-types.html[Action and connector types]. - -Some connectors that perform actions require less configuration. For example, you do not need to set the action frequency or variables for the {kibana-ref}/cases-action-type.html[Cases connector]. -==== -. After you select a connector, set its action frequency to define when notifications are sent: -+ -** **Summary of alerts**: Select this option to get a report that summarizes generated alerts, which you can review at your convenience. Alert summaries will be sent at the specified time intervals. -+ -[NOTE] -==== -When setting a custom notification frequency, do not choose a time that is shorter than the rule's execution schedule. -==== -** **For each alert**: Select this option to ensure notifications are sent every time new alerts are generated. -. (Optional) Specify additional conditions that need to be met for notifications to send. Click the toggle to enable a setting, then add the required details: -+ -** **If alert matches query**: Enter a KQL query that defines field-value pairs or query conditions that must be met for notifications to send. The query only searches alert documents in the indices specified for the rule. -** **If alert is generated during timeframe**: Set timeframe details. Notifications are only sent if alerts are generated within the timeframe you define. -. Complete the required connector type fields. Here is an example with {jira}: -+ -[role="screenshot"] -image::images/rules-ui-create/-detections-selected-action-type.png[] -. Use the default notification message or customize it. You can add more context to the message by clicking the icon above the message text box and selecting from a list of available <>. -. Create the rule with or without activation. -+ -[NOTE] -==== -When you activate a rule, it is queued, and its schedule is determined by -its initial run time. For example, if you activate a rule that runs every 5 -minutes at 14:03 but it does not run until 14:04, it will run again at 14:09. -==== - -[IMPORTANT] -==== -After you activate a rule, you can check if it is running as expected -using the <> on the Rules page. If you see -values in the `Gap` column, you can <>. - -When a rule fails to run, the {security-app} tries to rerun it at its next -scheduled run time. -==== - -[discrete] -[[rule-action-variables]] -=== Alert notification placeholders - -You can use http://mustache.github.io/[mustache syntax] to add variables to notification messages. The action frequency you choose determines the variables you can select from. - -The following variables can be passed for all rules: - -[NOTE] -==== -Refer to {kibana-ref}/rule-action-variables.html#alert-summary-action-variables[Action frequency: Summary of alerts] to learn about additional variables that can be passed if the rule's action frequency is **Summary of alerts**. -==== - -* `{{context.alerts}}`: Array of detected alerts -* `{{{context.results_link}}}`: URL to the alerts -* `{{context.rule.anomaly_threshold}}`: Anomaly threshold score above which -alerts are generated ({ml} rules only) -* `{{context.rule.description}}`: Rule description -* `{{context.rule.false_positives}}`: Rule false positives -* `{{context.rule.filters}}`: Rule filters (query rules only) -* `{{context.rule.id}}`: Unique rule ID returned after creating the rule -* `{{context.rule.index}}`: Indices rule runs on (query rules only) -* `{{context.rule.language}}`: Rule query language (query rules only) -* `{{context.rule.machine_learning_job_id}}`: ID of associated {ml} job ({ml} -rules only) -* `{{context.rule.max_signals}}`: Maximum allowed number of alerts per rule -execution -* `{{context.rule.name}}`: Rule name -* `{{context.rule.query}}`: Rule query (query rules only) -* `{{context.rule.references}}`: Rule references -* `{{context.rule.risk_score}}`: Default rule risk score -+ -[NOTE] -==== -This placeholder contains the rule's default values even when the **Risk score override** option is used. -==== -* `{{context.rule.rule_id}}`: Generated or user-defined rule ID that can be -used as an identifier across systems -* `{{context.rule.saved_id}}`: Saved search ID -* `{{context.rule.severity}}`: Default rule severity -+ -[NOTE] -==== -This placeholder contains the rule's default values even when the **Severity override** option is used. -==== -* `{{context.rule.threat}}`: Rule threat framework -* `{{context.rule.threshold}}`: Rule threshold values (threshold rules only) -* `{{context.rule.timeline_id}}`: Associated Timeline ID -* `{{context.rule.timeline_title}}`: Associated Timeline name -* `{{context.rule.type}}`: Rule type -* `{{context.rule.version}}`: Rule version -* `{{date}}`: Date the rule scheduled the action -* `{{kibanaBaseUrl}}`: Configured `server.publicBaseUrl` value, or empty string if not configured -* `{{rule.id}}`: ID of the rule -* `{{rule.name}}`: Name of the rule -* `{{rule.spaceId}}`: Space ID of the rule -* `{{rule.tags}}`: Tags of the rule -* `{{rule.type}}`: Type of rule -* `{{state.signals_count}}`: Number of alerts detected - -The following variables can only be passed if the rule’s action frequency is for each alert: - -* `{{alert.actionGroup}}`: Action group of the alert that scheduled actions for the rule -* `{{alert.actionGroupName}}`: Human-readable name of the action group of the alert that scheduled actions for the rule -* `{{alert.actionSubgroup}}`: Action subgroup of the alert that scheduled actions for the rule -* `{{alert.id}}`: ID of the alert that scheduled actions for the rule -* `{{alert.flapping}}`: A flag on the alert that indicates whether the alert status is changing repeatedly - -[discrete] -[[placeholder-examples]] -==== Alert placeholder examples - -To understand which fields to parse, see the {security-guide}/rule-api-overview.html[Detections API] to view the JSON representation of rules. - -// Link to classic docs until serverless API docs are available. - -Example using `{{context.rule.filters}}` to output a list of filters: - -[source,json] ----- -{{#context.rule.filters}} -{{^meta.disabled}}{{meta.key}} {{#meta.negate}}NOT {{/meta.negate}}{{meta.type}} {{^exists}}{{meta.value}}{{meta.params.query}}{{/exists}}{{/meta.disabled}} -{{/context.rule.filters}} ----- - -Example using `{{context.alerts}}` as an array, which contains each alert generated since the last time the action was executed: - -[source,json] ----- -{{#context.alerts}} -Detection alert for user: {{user.name}} -{{/context.alerts}} ----- - -Example using the mustache "current element" notation `{{.}}` to output all the rule references in the `signal.rule.references` array: - -[source,json] ----- -{{#signal.rule.references}} {{.}} {{/signal.rule.references}} ----- - -[discrete] -[[rule-response-action]] -=== Set up response actions (optional) - -Use response actions to set up additional functionality that will run whenever a rule executes: - -* **Osquery**: Include live Osquery queries with a custom query rule. When an alert is generated, Osquery automatically collects data on the system related to the alert. Refer to <> to learn more. -* **{elastic-defend}**: Automatically run response actions on an endpoint when rule conditions are met. For example, you can automatically isolate a host or terminate a process when specific activities or events are detected on the host. Refer to <> to learn more. - -[IMPORTANT] -==== -Host isolation involves quarantining a host from the network to prevent further spread of threats and limit potential damage. Be aware that automatic host isolation can cause unintended consequences, such as disrupting legitimate user activities or blocking critical business processes. -==== - -[discrete] -[[preview-rules]] -== Preview your rule (optional) - -You can preview any custom or prebuilt rule to find out how noisy it will be. For a custom rule, you can then adjust the rule's query or other settings. - -[NOTE] -==== -To preview rules, you must have the appropriate user role. Refer to <> for more information. -==== - -Click the **Rule preview** button while creating or editing a rule. The preview opens in a side panel, showing a histogram and table with the alerts you can expect, based on the defined rule settings and past events in your indices. - -[role="screenshot"] -image::images/rules-ui-create/-detections-preview-rule.png[Rule preview] - -The preview also includes the effects of rule exceptions and override fields. In the histogram, alerts are stacked by `event.category` (or `host.name` for machine learning rules), and alerts with multiple values are counted more than once. - -To interact with the rule preview: - -* Use the date and time picker to define the preview's time range. -+ -[TIP] -==== -Avoid setting long time ranges with short rule intervals, or the rule preview might time out. -==== -* Click **Refresh** to update the preview. -+ -** When you edit the rule's settings or the preview's time range, the button changes from blue to green to indicate that the rule has been edited since the last preview. -** For a relative time range (such as `Last 1 hour`), refresh the preview to check for the latest results. (Previews don't automatically refresh with new incoming data.) -* Click the **View details** icon (image:images/icons/expand.svg[View details]) in the alerts table to view the details of a particular alert. -* To resize the preview, hover between the rule settings and preview, then click and drag the border. You can also click the border, then the collapse icon (image:images/icons/menuRight.svg[Collapse menu]) to collapse and expand the preview. -* To close the preview, click the **Rule preview** button again. - -[discrete] -[[view-rule-es-queries]] -=== View your rule's {es} queries (optional) - -[NOTE] -==== -This option is only offered for {esql} and event correlation rules. -==== - -When previewing a rule, you can also learn about its {es} queries, which are submitted when the rule runs. This information can help you identify and troubleshoot potential rule issues. You can also use it to confirm that your rule is retrieving the expected data. - -To learn more about your rule's {es} queries, preview its results and do the following: - -. Select the **Show {es} requests, ran during rule executions** option below the preview's date and time picker. The **Preview logged results** section displays under the histogram and alerts table. -. Click the **Preview logged results** section to expand it. Within the section, each rule execution is shown on an individual row. -. Expand each row to learn more about the {es} queries that the rule submits each time it executes. The following details are provided: -+ -** When it started, and how long it took to complete -** A brief explanation of what the {es} queries do -** The actual {es} queries that the rule submits to indices containing events that are used during the rule execution -+ -[TIP] -==== -Run the queries in <> to determine if your rule is retrieving the expected data. For example, to test your rule’s exceptions, run the rule’s {es} queries, which will also contain exceptions added to the rule. If your rule’s exceptions are working as intended, the query will not return events that should be ignored. -====