From 9322ab2feadcdaf29b8a0cb41096cfaf6ce873aa Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 10:59:26 +0000 Subject: [PATCH 1/6] Initial plan From 502f69ec936b15602f530f8970f98aa421112f18 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 11:05:10 +0000 Subject: [PATCH 2/6] Add 120-day default retention period notes for ML jobs in log anomalies and log categories Co-authored-by: florent-leborgne <10208282+florent-leborgne@users.noreply.github.com> --- solutions/observability/logs/categorize-log-entries.md | 4 ++++ solutions/observability/logs/inspect-log-anomalies.md | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/solutions/observability/logs/categorize-log-entries.md b/solutions/observability/logs/categorize-log-entries.md index 4cabada421..04ad0407c8 100644 --- a/solutions/observability/logs/categorize-log-entries.md +++ b/solutions/observability/logs/categorize-log-entries.md @@ -27,6 +27,10 @@ Create a {{ml}} job to categorize log messages automatically. {{ml-cap}} observe 3. Add the indices that contain the logs you want to examine. By default, Machine Learning analyzes messages in all log indices that match the patterns set in the **logs sources** advanced setting. To open **Advanced settings**, find it in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). 4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results. +::::{note} +The {{ml}} job created for log categories has a default retention period of 120 days for results. This means that categorization results older than 120 days are automatically deleted to save storage space. You can change this retention period by modifying the `results_retention_days` setting for the job. +:::: + ## Analyze log categories [analyze-log-categories] diff --git a/solutions/observability/logs/inspect-log-anomalies.md b/solutions/observability/logs/inspect-log-anomalies.md index c697688688..be3e841852 100644 --- a/solutions/observability/logs/inspect-log-anomalies.md +++ b/solutions/observability/logs/inspect-log-anomalies.md @@ -32,6 +32,10 @@ Create a {{ml}} job to detect anomalous log entry rates automatically. 3. Add the indices that contain the logs you want to examine. By default, Machine Learning analyzes messages in all log indices that match the patterns set in the **logs source** advanced setting. To open **Advanced settings**, find it in the navigation menu or by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). 4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results. +::::{note} +The {{ml}} job created for log anomalies has a default retention period of 120 days for results. This means that anomaly detection results older than 120 days are automatically deleted to save storage space. You can change this retention period by modifying the `results_retention_days` setting for the job. +:::: + ## Anomalies chart [anomalies-chart] The Anomalies chart shows an overall, color-coded visualization of the log entry rate, partitioned according to the value of the Elastic Common Schema (ECS) [`event.dataset`](ecs://reference/ecs-event.md) field. This chart helps you quickly spot increases or decreases in each partition’s log rate. From d256405ab4f46b84871749752fa2b376feed341f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 12:33:04 +0000 Subject: [PATCH 3/6] Shorten retention notes and clarify exception in ML scale docs Co-authored-by: florent-leborgne <10208282+florent-leborgne@users.noreply.github.com> --- .../anomaly-detection/anomaly-detection-scale.md | 2 +- solutions/observability/logs/categorize-log-entries.md | 2 +- solutions/observability/logs/inspect-log-anomalies.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md index b4c3b4b60a..c366c599b3 100644 --- a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md +++ b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md @@ -100,7 +100,7 @@ See [Aggregating data for faster performance](ml-configuring-aggregation.md) to Set a results retention window to reduce the amount of results stored. -{{anomaly-detect-cap}} results are retained indefinitely by default. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. +{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} modules for logs which retain results for 120 days. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. ## 10. Optimize the renormalization window [renormalization-window] diff --git a/solutions/observability/logs/categorize-log-entries.md b/solutions/observability/logs/categorize-log-entries.md index 04ad0407c8..2a0c43868f 100644 --- a/solutions/observability/logs/categorize-log-entries.md +++ b/solutions/observability/logs/categorize-log-entries.md @@ -28,7 +28,7 @@ Create a {{ml}} job to categorize log messages automatically. {{ml-cap}} observe 4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results. ::::{note} -The {{ml}} job created for log categories has a default retention period of 120 days for results. This means that categorization results older than 120 days are automatically deleted to save storage space. You can change this retention period by modifying the `results_retention_days` setting for the job. +Log categorization {{ml}} jobs retain results for 120 days by default. Modify the `results_retention_days` setting to change this period. :::: diff --git a/solutions/observability/logs/inspect-log-anomalies.md b/solutions/observability/logs/inspect-log-anomalies.md index be3e841852..9095b260eb 100644 --- a/solutions/observability/logs/inspect-log-anomalies.md +++ b/solutions/observability/logs/inspect-log-anomalies.md @@ -33,7 +33,7 @@ Create a {{ml}} job to detect anomalous log entry rates automatically. 4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results. ::::{note} -The {{ml}} job created for log anomalies has a default retention period of 120 days for results. This means that anomaly detection results older than 120 days are automatically deleted to save storage space. You can change this retention period by modifying the `results_retention_days` setting for the job. +Log anomaly {{ml}} jobs retain results for 120 days by default. Modify the `results_retention_days` setting to change this period. :::: ## Anomalies chart [anomalies-chart] From 3eb64644d2e57a27eb53e6065453456cce6b728c Mon Sep 17 00:00:00 2001 From: florent-leborgne Date: Thu, 16 Oct 2025 14:40:06 +0200 Subject: [PATCH 4/6] Update explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md --- .../anomaly-detection/anomaly-detection-scale.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md index c366c599b3..7768b78729 100644 --- a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md +++ b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md @@ -100,7 +100,7 @@ See [Aggregating data for faster performance](ml-configuring-aggregation.md) to Set a results retention window to reduce the amount of results stored. -{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} modules for logs which retain results for 120 days. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. +{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} configurations for logs which retain results for 120 days. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. ## 10. Optimize the renormalization window [renormalization-window] From 3ce4bf139f4f44e22c809dde13fdc92a0ece76e3 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 16 Oct 2025 12:56:04 +0000 Subject: [PATCH 5/6] Add version tags for 9.2 and change modules to configurations Co-authored-by: florent-leborgne <10208282+florent-leborgne@users.noreply.github.com> --- .../anomaly-detection/anomaly-detection-scale.md | 2 +- solutions/observability/logs/categorize-log-entries.md | 2 ++ solutions/observability/logs/inspect-log-anomalies.md | 2 ++ 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md index 7768b78729..91181b12d4 100644 --- a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md +++ b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md @@ -100,7 +100,7 @@ See [Aggregating data for faster performance](ml-configuring-aggregation.md) to Set a results retention window to reduce the amount of results stored. -{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} configurations for logs which retain results for 120 days. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. +{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} configurations for logs {applies_to}`stack: ga 9.2` which retain results for 120 days. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. ## 10. Optimize the renormalization window [renormalization-window] diff --git a/solutions/observability/logs/categorize-log-entries.md b/solutions/observability/logs/categorize-log-entries.md index 2a0c43868f..c3c055a10d 100644 --- a/solutions/observability/logs/categorize-log-entries.md +++ b/solutions/observability/logs/categorize-log-entries.md @@ -28,6 +28,8 @@ Create a {{ml}} job to categorize log messages automatically. {{ml-cap}} observe 4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results. ::::{note} +:applies_to: stack: ga 9.2 + Log categorization {{ml}} jobs retain results for 120 days by default. Modify the `results_retention_days` setting to change this period. :::: diff --git a/solutions/observability/logs/inspect-log-anomalies.md b/solutions/observability/logs/inspect-log-anomalies.md index 9095b260eb..6950e5fbec 100644 --- a/solutions/observability/logs/inspect-log-anomalies.md +++ b/solutions/observability/logs/inspect-log-anomalies.md @@ -33,6 +33,8 @@ Create a {{ml}} job to detect anomalous log entry rates automatically. 4. Click **Create ML job**. This creates and runs the job. It takes a few minutes for the {{ml}} robots to collect the necessary data. After the job has processed the data, you can view its results. ::::{note} +:applies_to: stack: ga 9.2 + Log anomaly {{ml}} jobs retain results for 120 days by default. Modify the `results_retention_days` setting to change this period. :::: From e5d6ddb9c557242546fac51984832f4e6e7d8bd4 Mon Sep 17 00:00:00 2001 From: florent-leborgne Date: Thu, 16 Oct 2025 14:59:35 +0200 Subject: [PATCH 6/6] Update explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md --- .../anomaly-detection/anomaly-detection-scale.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md index 91181b12d4..6c13eb541e 100644 --- a/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md +++ b/explore-analyze/machine-learning/anomaly-detection/anomaly-detection-scale.md @@ -100,7 +100,7 @@ See [Aggregating data for faster performance](ml-configuring-aggregation.md) to Set a results retention window to reduce the amount of results stored. -{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} configurations for logs {applies_to}`stack: ga 9.2` which retain results for 120 days. Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. +{{anomaly-detect-cap}} results are retained indefinitely by default, except for predefined {{ml}} configurations for logs which retain results for 120 days ({applies_to}`stack: ga 9.2`). Results build up over time, and your result index may be quite large. A large results index is slow to query and takes up significant space on your cluster. Consider how long you wish to retain the results and set `results_retention_days` accordingly – for example, to 30 or 60 days – to avoid unnecessarily large result indices. Deleting old results does not affect the model behavior. You can change this setting for existing jobs. ## 10. Optimize the renormalization window [renormalization-window]