diff --git a/docs/detections/rules-ui-monitor.asciidoc b/docs/detections/rules-ui-monitor.asciidoc index 2e96c654be..587982e0e4 100644 --- a/docs/detections/rules-ui-monitor.asciidoc +++ b/docs/detections/rules-ui-monitor.asciidoc @@ -112,65 +112,48 @@ image::images/timestamp-override.png[] [[ml-job-compatibility]] ==== Troubleshoot missing alerts for {ml} jobs -The <> have dependencies on data fields -that are populated by {beats} and {agent} integrations. In version 7.11, new -{ml} jobs (<> and <>) were provided, -which operate on newer ECS fields than the previous Security: {winlogbeat} and -Security: {auditbeat} jobs. However, the <> were -not updated to use the new {ml} jobs. - -Therefore: - -* If you have only 7.10 or earlier versions of {beats}, you can continue using -the Security:Auditbeat and Security:Winlogbeat {ml} jobs and the prebuilt {ml} -rules that have been in the {security-app} since version 7.5. -* If you have only 7.11 or later versions of {beats}, use the Security:Linux and -Security:Windows {ml} jobs. If you want to generate alerts for anomalies in -these jobs, make clones of the existing {ml} rules and update them to use the -new jobs. -* If you have a mix of old and new versions of {beats} or you have a mix of -{beats} and {elastic-endpoint} integrations, use both the old and new {ml} jobs. -If you want alerts for anomalies in the new jobs, make clones of the existing -{ml} rules and update them to use the new jobs. -* If you have a non-Elastic data shipper that gathers ECS-compatible Windows -events, use the Security:Windows {ml} jobs. If you want alerts for anomalies in -these jobs, make clones of the existing {ml} rules and update them to use these -jobs. - -If you are cloning prebuilt {ml} rules to generate alerts for the new {ml} jobs, -the following rules are affected: - -* <>: Use -`v2_linux_anomalous_network_port_activity_ecs` instead of -`linux_anomalous_network_port_activity_ecs`. -* <>: Use -`v2_linux_anomalous_process_all_hosts_ecs` instead of -`linux_anomalous_process_all_hosts_ecs`. -* <>: Use `v2_linux_anomalous_user_name_ecs` instead of -`linux_anomalous_user_name_ecs`. -* <>: Use -`v2_linux_rare_metadata_process` instead of `linux_rare_metadata_process`. -* <>: Use -`v2_linux_rare_metadata_user` instead of `linux_rare_metadata_user`. -* <>: Use `v2_rare_process_by_host_linux_ecs` -instead of `rare_process_by_host_linux_ecs`. -* <>: Use -`v2_rare_process_by_host_windows_ecs` instead of -`rare_process_by_host_windows_ecs`. -* <>: Use -`v2_windows_anomalous_network_activity_ecs` instead of -`windows_anomalous_network_activity_ecs`. -* <>: Use `v2_windows_anomalous_path_activity_ecs` -instead of `windows_anomalous_path_activity_ecs`. -* <>: Use -`v2_windows_anomalous_process_creation` instead of -`windows_anomalous_process_creation`. -* <>: Use -`v2_windows_anomalous_process_all_hosts_ecs` instead of -`windows_anomalous_process_all_hosts_ecs`. -* <>: Use `v2_windows_anomalous_user_name_ecs` instead -of `windows_anomalous_user_name_ecs`. -* <>: Use -`v2_windows_rare_metadata_process` instead of `windows_rare_metadata_process`. -* <>: Use -`v2_windows_rare_metadata_user` instead of `windows_rare_metadata_user`. +{ml-cap} detection rules use {ml} jobs that have dependencies on data fields populated by the {beats} and {agent} integrations. In {stack} version 8.3, new {ml} jobs (prefixed with `v3`) were released to operate on the ECS fields available at that time. + +If you're using 8.2 or earlier versions of {beats} or {agent} with {stack} version 8.3 or later, you may need to duplicate prebuilt rules or create new custom rules _before_ you update the Elastic prebuilt rules. Once you update the prebuilt rules, they will only use `v3` {ml} jobs. Duplicating the relevant prebuilt rules before updating them ensures continued coverage by allowing you to keep using `v1` or `v2` jobs (in the duplicated rules) while also running the new `v3` jobs (in the updated prebuilt rules). + +[IMPORTANT] +===== +* Duplicated rules may result in duplicate anomaly detections and alerts. +* Ensure that the relevant `v3` {ml} jobs are running before you update the Elastic prebuilt rules. +===== + +* If you only have *8.3 or later versions of {beats} and {agent}*: You can download or update your prebuilt rules and use the latest `v3` {ml} jobs. No additional action is required. + +* If you only have *8.2 or earlier versions of {beats} or {agent}*, or *a mix of old and new versions*: To continue using the `v1` and `v2` {ml} jobs specified by pre-8.3 prebuilt detection rules, you must duplicate affected prebuilt rules _before_ updating them to the latest rule versions. The duplicated rules can continue using the same `v1` and `v2` {ml} jobs, and the updated prebuilt {ml} rules will use the new `v3` {ml} jobs. + +* If you have *a non-Elastic data shipper that gathers ECS-compatible events*: You can use the latest `v3` {ml} jobs with no additional action required, as long as your data shipper uses the latest ECS specifications. However, if you're migrating from {ml} rules using `v1`/`v2` jobs, ensure that you start the relevant `v3` jobs before updating the Elastic prebuilt rules. + +The following Elastic prebuilt rules use the new `v3` {ml} jobs to generate alerts. Duplicate their associated `v1`/`v2` prebuilt rules _before_ updating them if you need continued coverage from the `v1`/`v2` {ml} jobs: + +* <>: `v3_linux_anomalous_network_port_activity` + +* <>: `v3_linux_anomalous_process_all_hosts` + +* <>: `v3_linux_anomalous_user_name` + +* <>: `v3_linux_rare_metadata_process` + +* <>: `v3_linux_rare_metadata_user` + +* <>: `v3_rare_process_by_host_linux` + +* <>: `v3_rare_process_by_host_windows` + +* <>: `v3_windows_anomalous_network_activity` + +* <>: `v3_windows_anomalous_path_activity` + +* <>: `v3_windows_anomalous_process_creation` + +* <>: `v3_windows_anomalous_process_all_hosts` + +* <>: `v3_windows_anomalous_user_name` + +* <>: `v3_windows_rare_metadata_process` + +* <>: `v3_windows_rare_metadata_user`