From 5b049387162e9641abf5257fdfd8bda1ee63de07 Mon Sep 17 00:00:00 2001 From: Andrei Dan Date: Tue, 3 Oct 2023 16:13:53 +0100 Subject: [PATCH 1/2] Document ILM waits for tsds end_time to lapse in some actions --- .../data-streams/downsampling-ilm.asciidoc | 12 ++++++++++-- .../data-streams/tsds-index-settings.asciidoc | 4 ++++ docs/reference/data-streams/tsds.asciidoc | 14 ++++++++++---- 3 files changed, 24 insertions(+), 6 deletions(-) diff --git a/docs/reference/data-streams/downsampling-ilm.asciidoc b/docs/reference/data-streams/downsampling-ilm.asciidoc index 58fa15a56aca9..cfc38ca719379 100644 --- a/docs/reference/data-streams/downsampling-ilm.asciidoc +++ b/docs/reference/data-streams/downsampling-ilm.asciidoc @@ -37,8 +37,16 @@ To enable downsampling, add a <> and set which you want to aggregate the original time series data. In this example, an ILM policy is configured for the `hot` phase. The downsample -takes place after the initial index rollover, which for demonstration -purposes is set to run after five minutes. +takes place after the index is rolled over and the <> +has lapsed as the source index is still expected to receive writes until then. +{ilm-cap} will not proceed with any action that expects the index to not receive +writes anymore until the <> has +passed. The {ilm-cap} actions that wait on the end time before proceeding are: +- <> +- <> +- <> +- <> +- <> [source,console] ---- diff --git a/docs/reference/data-streams/tsds-index-settings.asciidoc b/docs/reference/data-streams/tsds-index-settings.asciidoc index 8091163ffe883..c0cae9e365114 100644 --- a/docs/reference/data-streams/tsds-index-settings.asciidoc +++ b/docs/reference/data-streams/tsds-index-settings.asciidoc @@ -33,6 +33,10 @@ days). Only indices with an `index.mode` of `time_series` support this setting. For more information, refer to <>. Additionally this setting can not be less than `time_series.poll_interval` cluster setting. +NOTE: Increasing the `look_ahead_time` will also increase the amount of time {ilm-cap} +waits before being able to proceed with executing the actions that expect the +index to not receive any writes anymore. For more information, refer to <>. + [[index-look-back-time]] `index.look_back_time`:: (<<_static_index_settings,Static>>, <>) diff --git a/docs/reference/data-streams/tsds.asciidoc b/docs/reference/data-streams/tsds.asciidoc index 9c3151a509b14..75c2b6e858b87 100644 --- a/docs/reference/data-streams/tsds.asciidoc +++ b/docs/reference/data-streams/tsds.asciidoc @@ -218,10 +218,16 @@ if the index isn't the most recent backing index. image::images/data-streams/time-bound-indices.svg[align="center"] -TIP: Some {ilm-init} actions, such as <>, -<>, and <>, -make a backing index read-only. You cannot add documents to read-only indices. -Keep this in mind when defining the index lifecycle policy for your TSDS. +TIP: Some {ilm-init} actions mark the source index as read-only, or expect the index +to not be actively written anymore in order to provide good performance. These actions are: +- <> +- <> +- <> +- <> +- <> +{ilm-cap} will **not** proceed with executing these actions until the upper time-bound +for accepting writes, represented by the <> +index setting, has lapsed. If no backing index can accept a document's `@timestamp` value, {es} rejects the document. From 297d24dc2fc3dc7e67eeeae3bb56d847a6fb93ed Mon Sep 17 00:00:00 2001 From: Andrei Dan Date: Wed, 4 Oct 2023 12:39:04 +0100 Subject: [PATCH 2/2] Docs update --- docs/reference/data-streams/downsampling-ilm.asciidoc | 3 ++- docs/reference/data-streams/tsds.asciidoc | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/reference/data-streams/downsampling-ilm.asciidoc b/docs/reference/data-streams/downsampling-ilm.asciidoc index cfc38ca719379..94990040d79f1 100644 --- a/docs/reference/data-streams/downsampling-ilm.asciidoc +++ b/docs/reference/data-streams/downsampling-ilm.asciidoc @@ -38,10 +38,11 @@ which you want to aggregate the original time series data. In this example, an ILM policy is configured for the `hot` phase. The downsample takes place after the index is rolled over and the <> -has lapsed as the source index is still expected to receive writes until then. +has lapsed as the source index is still expected to receive major writes until then. {ilm-cap} will not proceed with any action that expects the index to not receive writes anymore until the <> has passed. The {ilm-cap} actions that wait on the end time before proceeding are: +- <> - <> - <> - <> diff --git a/docs/reference/data-streams/tsds.asciidoc b/docs/reference/data-streams/tsds.asciidoc index 75c2b6e858b87..d5ffce309a5aa 100644 --- a/docs/reference/data-streams/tsds.asciidoc +++ b/docs/reference/data-streams/tsds.asciidoc @@ -220,6 +220,7 @@ image::images/data-streams/time-bound-indices.svg[align="center"] TIP: Some {ilm-init} actions mark the source index as read-only, or expect the index to not be actively written anymore in order to provide good performance. These actions are: +- <> - <> - <> - <>