From a00a352a955b56541abffc263bf9174b77b63eb0 Mon Sep 17 00:00:00 2001 From: Felix Barnsteiner Date: Tue, 30 Sep 2025 11:34:59 +0200 Subject: [PATCH 1/3] Document index.dimensions-based routing --- .../time-series-data-stream-tsds.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/manage-data/data-store/data-streams/time-series-data-stream-tsds.md b/manage-data/data-store/data-streams/time-series-data-stream-tsds.md index cd673edbf0..aefe24a406 100644 --- a/manage-data/data-store/data-streams/time-series-data-stream-tsds.md +++ b/manage-data/data-store/data-streams/time-series-data-stream-tsds.md @@ -198,12 +198,27 @@ You can use the [get data stream API](https://www.elastic.co/docs/api/doc/elasti ### Dimension-based routing [dimension-based-routing] -Within each TSDS backing index, {{es}} uses the [`index.routing_path`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-routing-path) index setting to route documents with the same dimensions to the same shards. +Within each TSDS backing index, {{es}} uses one of two strategies to route documents with the same dimensions to the same shards. -When you create the matching index template for a TSDS, you must specify one or more dimensions in the `index.routing_path` setting. Each document in a TSDS must contain one or more dimensions that match the `index.routing_path` setting. +1. Based on the internally managed `index.dimensions` index setting (available as of stack version 9.2) +2. Based on the [`index.routing_path`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-routing-path) index setting + +The `index.dimensions`-based strategy uses a list of dimension paths that's automatically kept up-to-date and is not user-configurable. +When active, the routing decision will be made based on all dimension fields instead of only those defined statically via the `index.routing_path` setting, +which can help to prevent shard hot-spotting issues. +This strategy is only available on data streams and requires that there are no dynamic templates that set `time_series_dimension: true`. +It is used by default if applicable and improves the ingest performance as dimensions only need to be processed once for the purposes of routing and creating the `_tsid` field. + +For more details including how to disable this strategy, +see [`index.index_dimensions_tsid_strategy_enabled`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-dimensions-tsid-strategy-enabled) + +To use the `index.routing_path`-based strategy, +you must specify one or more dimensions in the `index.routing_path` setting. Each document in a TSDS must contain one or more dimensions that match the `index.routing_path` setting. The `index.routing_path` setting accepts wildcard patterns (for example `dim.*`) and can dynamically match new fields. However, {{es}} will reject any mapping updates that add scripted, runtime, or non-dimension fields that match the `index.routing_path` value. +The following applies to both strategies: + [Pass-through](elasticsearch://reference/elasticsearch/mapping-reference/passthrough.md#passthrough-dimensions) fields may be configured as dimension containers. In this case, their sub-fields get included to the routing path automatically. TSDS documents don’t support a custom `_routing` value. Similarly, you can’t require a `_routing` value in mappings for a TSDS. From 86e5bd319f1390548d95b3924649bb814168c29e Mon Sep 17 00:00:00 2001 From: Felix Barnsteiner Date: Tue, 30 Sep 2025 14:58:54 +0200 Subject: [PATCH 2/3] Apply suggestions from code review Co-authored-by: Kostas Krikellas <131142368+kkrik-es@users.noreply.github.com> --- .../data-store/data-streams/time-series-data-stream-tsds.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/manage-data/data-store/data-streams/time-series-data-stream-tsds.md b/manage-data/data-store/data-streams/time-series-data-stream-tsds.md index aefe24a406..9ebb4407dd 100644 --- a/manage-data/data-store/data-streams/time-series-data-stream-tsds.md +++ b/manage-data/data-store/data-streams/time-series-data-stream-tsds.md @@ -198,15 +198,15 @@ You can use the [get data stream API](https://www.elastic.co/docs/api/doc/elasti ### Dimension-based routing [dimension-based-routing] -Within each TSDS backing index, {{es}} uses one of two strategies to route documents with the same dimensions to the same shards. +Within each TSDS backing index, {{es}} uses one of the following strategies to route documents with the same dimensions to the same shards. 1. Based on the internally managed `index.dimensions` index setting (available as of stack version 9.2) 2. Based on the [`index.routing_path`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-routing-path) index setting The `index.dimensions`-based strategy uses a list of dimension paths that's automatically kept up-to-date and is not user-configurable. -When active, the routing decision will be made based on all dimension fields instead of only those defined statically via the `index.routing_path` setting, +When active, the routing decision will be made based on all dimension fields, instead of only those defined statically via the `index.routing_path` setting, which can help to prevent shard hot-spotting issues. -This strategy is only available on data streams and requires that there are no dynamic templates that set `time_series_dimension: true`. +This strategy is only available on data streams with no dynamic templates setting `time_series_dimension: true`. It is used by default if applicable and improves the ingest performance as dimensions only need to be processed once for the purposes of routing and creating the `_tsid` field. For more details including how to disable this strategy, From 65fc86ee91f4e84ec38ee3f79b3422948cf8f334 Mon Sep 17 00:00:00 2001 From: Felix Barnsteiner Date: Tue, 30 Sep 2025 15:04:13 +0200 Subject: [PATCH 3/3] Clarify routing strategies for TSDS backing indices in documentation --- .../data-streams/time-series-data-stream-tsds.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/manage-data/data-store/data-streams/time-series-data-stream-tsds.md b/manage-data/data-store/data-streams/time-series-data-stream-tsds.md index 9ebb4407dd..d0fdbb51a1 100644 --- a/manage-data/data-store/data-streams/time-series-data-stream-tsds.md +++ b/manage-data/data-store/data-streams/time-series-data-stream-tsds.md @@ -200,17 +200,16 @@ You can use the [get data stream API](https://www.elastic.co/docs/api/doc/elasti Within each TSDS backing index, {{es}} uses one of the following strategies to route documents with the same dimensions to the same shards. -1. Based on the internally managed `index.dimensions` index setting (available as of stack version 9.2) -2. Based on the [`index.routing_path`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-routing-path) index setting +1. Based on the internally managed `index.dimensions` index setting (preferred). Available as of stack version 9.2. +2. Based on the [`index.routing_path`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-routing-path) index setting (as a fallback). The `index.dimensions`-based strategy uses a list of dimension paths that's automatically kept up-to-date and is not user-configurable. -When active, the routing decision will be made based on all dimension fields, instead of only those defined statically via the `index.routing_path` setting, -which can help to prevent shard hot-spotting issues. +When active, the routing decision will be made based on all dimension fields, instead of only those defined statically via the `index.routing_path` setting. This strategy is only available on data streams with no dynamic templates setting `time_series_dimension: true`. It is used by default if applicable and improves the ingest performance as dimensions only need to be processed once for the purposes of routing and creating the `_tsid` field. -For more details including how to disable this strategy, -see [`index.index_dimensions_tsid_strategy_enabled`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-dimensions-tsid-strategy-enabled) +This strategy can be disabled by setting [`index.index_dimensions_tsid_strategy_enabled`](elasticsearch://reference/elasticsearch/index-settings/time-series.md#index-dimensions-tsid-strategy-enabled) to `false`, +or by manually setting `index.routing_path`. To use the `index.routing_path`-based strategy, you must specify one or more dimensions in the `index.routing_path` setting. Each document in a TSDS must contain one or more dimensions that match the `index.routing_path` setting.