From bece4774faa94438d36f641dbe677a876e5b1e5a Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Tue, 12 Aug 2025 16:47:48 -0400 Subject: [PATCH 1/7] Clarify scope and procedure for 'Configure a lifecycle policy' --- .../configure-lifecycle-policy.md | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index 36517b9ea4..ad2caa9124 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -10,22 +10,23 @@ products: # Configure a lifecycle policy [set-up-lifecycle-policy] -An [{{ilm}}](/manage-data/lifecycle/index-lifecycle-management.md) ({{ilm-init}}) policy defines how your indices are managed over time, automating when and how they transition as they age. +An [{{ilm}}](/manage-data/lifecycle/index-lifecycle-management.md) ({{ilm-init}}) policy defines how your indices are managed over time, automating when and how they transition as they age. You can use {{ilm-init}} to manage both data streams and standalone indices. When used with the latter, some additional configuration is required compared to the more automated process for data streams. This page describes the steps to configure an {{ilm-init}} lifecycle policy for both scenarios. + +:::{note} +This page is specifically about using {{ilm-init}} with indices or data streams. If you're looking for a simpler, less feature-rich lifecycle management option for data streams only, refer to [Data stream lifecycle](/manage-data/lifecycle/data-stream.md). Check [Data lifecycle](/manage-data/lifecycle.md) to compare these lifecycle management options. +::: + **Consider these aspects when configuring an {{ilm-init}} policy:** -* For {{ilm-init}} to manage an index, you need to specify a valid policy in the `index.lifecycle.name` index setting. +* For {{ilm-init}} to manage an index or data stream, you need to specify a valid policy in the `index.lifecycle.name` index setting. -* To configure a lifecycle policy for [rolling indices](rollover.md) or data streams, you create the policy and add it to the [index template](../../data-store/templates.md). +* To configure a lifecycle policy for [rolling indices](rollover.md) or data streams, you create the policy and add it to the [index template](../../data-store/templates.md). Data streams are generally recommended in favor of rolling indices due to the lesser amount of manual configuration required. When you use {{ilm-init}} with rolling indices, you need to perform the additional steps of creating an initial managed index (ensuring that it is named appropriately) and assigning an alias to it, as described in [Step 3](#create-initial-index) on this page. * To use a policy to manage a single index, you can specify a lifecycle policy when you create the index, or apply a policy directly to an existing index. * {{ilm-init}} policies are stored in the global cluster state and can be included in snapshots by setting `include_global_state` to `true` when you [take the snapshot](../../../deploy-manage/tools/snapshot-and-restore/create-snapshots.md). When the snapshot is restored, all of the policies in the global state are restored and any local policies with the same names are overwritten. -:::{note} -This page is specifically about {{ilm-init}}. If you're looking for a simpler lifecycle management option for data streams, refer to [Data stream lifecycle](/manage-data/lifecycle/data-stream.md). Check [Data lifecycle](/manage-data/lifecycle.md) to compare these lifecycle management options. -::: - ## Overview [ilm-configure-overview] To set up ILM to manage one or more indices, the general procedure is as follows: @@ -35,7 +36,7 @@ To set up ILM to manage one or more indices, the general procedure is as follows If you're configuring ILM for rolling indices and not using [data streams](../../data-store/data-streams.md), you additionally need to: -3. [Create an initial managed index](#create-initial-index) +3. [Create an initial managed index and alias](#create-initial-index) You can perform these actions in {{kib}} or using the {{es}} API. @@ -207,7 +208,7 @@ PUT _index_template/my_template ::: :::: -## Create an initial managed index [create-initial-index] +## Create an initial managed index and alias [create-initial-index] When you set up policies for your own rolling indices and are not using the recommended [data streams](../../data-store/data-streams.md), you must manually create the first index managed by a policy and designate it as the write index. @@ -250,7 +251,7 @@ Now you can start indexing data to the rollover alias specified in the lifecycle :sync: api Use the [Create an index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create) to create the initial managed index. -The following request creates the `test-000001` index. Because it matches the index pattern specified in `my_template`, {{es}} automatically applies the settings from that template. +The following request creates the `test-000001` index, with the alias `test-alias`. Because the index name matches the index pattern specified in `my_template`, {{es}} automatically applies the settings from that template. ```console PUT test-000001 From 6b0483de6bd7aae227a740791f06fb3e54dc2d4c Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Tue, 12 Aug 2025 16:59:08 -0400 Subject: [PATCH 2/7] A bit more... --- .../index-lifecycle-management/configure-lifecycle-policy.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index ad2caa9124..fed8b38b6c 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -16,7 +16,6 @@ An [{{ilm}}](/manage-data/lifecycle/index-lifecycle-management.md) ({{ilm-init}} This page is specifically about using {{ilm-init}} with indices or data streams. If you're looking for a simpler, less feature-rich lifecycle management option for data streams only, refer to [Data stream lifecycle](/manage-data/lifecycle/data-stream.md). Check [Data lifecycle](/manage-data/lifecycle.md) to compare these lifecycle management options. ::: - **Consider these aspects when configuring an {{ilm-init}} policy:** * For {{ilm-init}} to manage an index or data stream, you need to specify a valid policy in the `index.lifecycle.name` index setting. @@ -214,6 +213,8 @@ When you set up policies for your own rolling indices and are not using the reco The name of the index must match the pattern defined in the index template and end with a number. This number is incremented to generate the name of indices created by the rollover action. +This step is not required when you're using data streams. + ::::{important} When you enable {{ilm}} for {{beats}}, {{agent}}, or for the {{agent}} or {{ls}} {{es}} output plugins, the necessary policies and configuration changes are applied automatically. If you'd like to create a specialized ILM policy for any data stream, refer to our tutorial [Customize built-in policies](/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md). :::: From 4635067d1c7042eed6675edc48ec6201f7077ed4 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Thu, 14 Aug 2025 10:55:03 -0400 Subject: [PATCH 3/7] fix up 'not for data streams' text --- .../index-lifecycle-management/configure-lifecycle-policy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index fed8b38b6c..33a7e4ee62 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -213,7 +213,7 @@ When you set up policies for your own rolling indices and are not using the reco The name of the index must match the pattern defined in the index template and end with a number. This number is incremented to generate the name of indices created by the rollover action. -This step is not required when you're using data streams. +This step is required only when you're planning to use {{ilm-init}} with rolling indices. It is not required when you're using data streams, where the initial managed index is created automatically. ::::{important} When you enable {{ilm}} for {{beats}}, {{agent}}, or for the {{agent}} or {{ls}} {{es}} output plugins, the necessary policies and configuration changes are applied automatically. If you'd like to create a specialized ILM policy for any data stream, refer to our tutorial [Customize built-in policies](/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies.md). From 4653a778f10edaf2018047553d773a655d85316b Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Fri, 22 Aug 2025 13:28:04 -0400 Subject: [PATCH 4/7] fixup --- .../index-lifecycle-management/configure-lifecycle-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index 33a7e4ee62..0a60132d58 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -1,5 +1,5 @@ --- -navigation_title: Configure a policy +navigation_title: Configure a lifecycle policy mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html applies_to: @@ -20,7 +20,7 @@ This page is specifically about using {{ilm-init}} with indices or data streams. * For {{ilm-init}} to manage an index or data stream, you need to specify a valid policy in the `index.lifecycle.name` index setting. -* To configure a lifecycle policy for [rolling indices](rollover.md) or data streams, you create the policy and add it to the [index template](../../data-store/templates.md). Data streams are generally recommended in favor of rolling indices due to the lesser amount of manual configuration required. When you use {{ilm-init}} with rolling indices, you need to perform the additional steps of creating an initial managed index (ensuring that it is named appropriately) and assigning an alias to it, as described in [Step 3](#create-initial-index) on this page. +* To configure a lifecycle policy for [rolling indices](rollover.md) or data streams, you create the policy and add it to the [index template](../../data-store/templates.md). Data streams are generally recommended in favor of rolling indices due to the lesser amount of manual configuration required. When you use {{ilm-init}} with rolling indices, you must, additionally, create an initial managed index (ensuring that it is named appropriately) and assign an alias to it. This additional process is described in [Step 3](#create-initial-index) on this page. * To use a policy to manage a single index, you can specify a lifecycle policy when you create the index, or apply a policy directly to an existing index. From fa45b8fa1aeeb309020b7fad95040f851124f56d Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Thu, 4 Sep 2025 09:33:57 -0400 Subject: [PATCH 5/7] Update manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md Co-authored-by: Vlada Chirmicci --- .../index-lifecycle-management/configure-lifecycle-policy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index 0a60132d58..a4645fb437 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -10,7 +10,7 @@ products: # Configure a lifecycle policy [set-up-lifecycle-policy] -An [{{ilm}}](/manage-data/lifecycle/index-lifecycle-management.md) ({{ilm-init}}) policy defines how your indices are managed over time, automating when and how they transition as they age. You can use {{ilm-init}} to manage both data streams and standalone indices. When used with the latter, some additional configuration is required compared to the more automated process for data streams. This page describes the steps to configure an {{ilm-init}} lifecycle policy for both scenarios. +An [{{ilm}}](/manage-data/lifecycle/index-lifecycle-management.md) ({{ilm-init}}) policy defines how your indices are managed over time, automating when and how they transition as they age. You can use {{ilm-init}} to manage both indices and [data streams](/manage-data/data-store/data-streams.md). There are fewer configuration steps required to set up ILM with data streams. In comparison, configuring ILM with indices requires you to create an initial managed index and alias in addition to defining a policy and creating a template to apply it. This page describes the steps to configure an {{ilm-init}} lifecycle policy for both scenarios. :::{note} This page is specifically about using {{ilm-init}} with indices or data streams. If you're looking for a simpler, less feature-rich lifecycle management option for data streams only, refer to [Data stream lifecycle](/manage-data/lifecycle/data-stream.md). Check [Data lifecycle](/manage-data/lifecycle.md) to compare these lifecycle management options. From 4365ca43a580a59854280d64667220dcc2effe4a Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Thu, 4 Sep 2025 09:34:15 -0400 Subject: [PATCH 6/7] Update manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md Co-authored-by: Vlada Chirmicci --- .../index-lifecycle-management/configure-lifecycle-policy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index a4645fb437..dcf07f09c2 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -13,7 +13,7 @@ products: An [{{ilm}}](/manage-data/lifecycle/index-lifecycle-management.md) ({{ilm-init}}) policy defines how your indices are managed over time, automating when and how they transition as they age. You can use {{ilm-init}} to manage both indices and [data streams](/manage-data/data-store/data-streams.md). There are fewer configuration steps required to set up ILM with data streams. In comparison, configuring ILM with indices requires you to create an initial managed index and alias in addition to defining a policy and creating a template to apply it. This page describes the steps to configure an {{ilm-init}} lifecycle policy for both scenarios. :::{note} -This page is specifically about using {{ilm-init}} with indices or data streams. If you're looking for a simpler, less feature-rich lifecycle management option for data streams only, refer to [Data stream lifecycle](/manage-data/lifecycle/data-stream.md). Check [Data lifecycle](/manage-data/lifecycle.md) to compare these lifecycle management options. +This page is specifically about using {{ilm-init}} with indices or data streams. If you're looking for a simpler data streams lifecycle management option that focuses on a data retention period, refer to [Data stream lifecycle](/manage-data/lifecycle/data-stream.md). Check [Data lifecycle](/manage-data/lifecycle.md) to compare these lifecycle management options. ::: **Consider these aspects when configuring an {{ilm-init}} policy:** From 595b0a17657bf1c5101cf1f4b8f6087bc584bdd5 Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Thu, 4 Sep 2025 09:34:26 -0400 Subject: [PATCH 7/7] Update manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md Co-authored-by: Vlada Chirmicci --- .../index-lifecycle-management/configure-lifecycle-policy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md index dcf07f09c2..3921e36994 100644 --- a/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md +++ b/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy.md @@ -18,7 +18,7 @@ This page is specifically about using {{ilm-init}} with indices or data streams. **Consider these aspects when configuring an {{ilm-init}} policy:** -* For {{ilm-init}} to manage an index or data stream, you need to specify a valid policy in the `index.lifecycle.name` index setting. +* To manage an index or data stream with {{ilm-init}}, you need to specify a valid policy in the `index.lifecycle.name` index setting. * To configure a lifecycle policy for [rolling indices](rollover.md) or data streams, you create the policy and add it to the [index template](../../data-store/templates.md). Data streams are generally recommended in favor of rolling indices due to the lesser amount of manual configuration required. When you use {{ilm-init}} with rolling indices, you must, additionally, create an initial managed index (ensuring that it is named appropriately) and assign an alias to it. This additional process is described in [Step 3](#create-initial-index) on this page.