Datastream temporal_ranges is wrong in presence of downsampling

[lkts]

Elasticsearch Version

8.14

Installed Plugins

No response

Java Version

bundled

OS Version

all

Problem Description

Original issue i noticed - every execution of a request like GET /_data_stream/my-data-stream for TSDS with downsampling enabled produces a warning of format previous backing index [..] range [..] range is colliding with current backing [..] index range [..].

While looking at this issue i realized that it looks like temporal_ranges information for data stream is wrong. See logs section for details.

Additionally the log is misleading - information it provides is incorrect, start and end times of indices reported are not actual start and end time for this index.

Steps to Reproduce

1. Create a TSDS with downsampling enabled
2. Create at least two downsampled indices using rollover or ILM

Logs (if relevant)

Example

previous backing index [downsample-5m-.ds-my-data-stream-2024.03.27-000002] range [2024-03-27T21:34:37Z/2024-03-27T22:19:05Z] range is colliding with current backing [downsample-5m-.ds-my-data-stream-2024.03.27-000001] index range [2024-03-27T19:04:37Z/2024-03-27T21:34:37Z]

Datastream information (relevant parts):

"indices" : [
        {
          "index_name" : "downsample-5m-.ds-my-data-stream-2024.03.27-000002",
          "index_uuid" : "i40UG4WGSa-EEgqlxmmQKA",
          "prefer_ilm" : true,
          "managed_by" : "Data stream lifecycle"
        },
        {
          "index_name" : "downsample-5m-.ds-my-data-stream-2024.03.27-000001",
          "index_uuid" : "3PozqIHaQ-W1vJkNZAX3bA",
          "prefer_ilm" : true,
          "managed_by" : "Data stream lifecycle"
        },
        {
          "index_name" : ".ds-my-data-stream-2024.03.27-000003",
          "index_uuid" : "Et2kbilgRUi13_kIh6md0w",
          "prefer_ilm" : true,
          "managed_by" : "Data stream lifecycle"
        },
        {
          "index_name" : ".ds-my-data-stream-2024.03.28-000008",
          "index_uuid" : "VKhQEQekSFKxgexV_xOFeQ",
          "prefer_ilm" : true,
          "managed_by" : "Data stream lifecycle"
        }
      ],
............
"time_series" : {
        "temporal_ranges" : [
          {
            "start" : "2024-03-27T21:34:37.000Z",
            "end" : "2024-03-28T19:44:39.000Z"
          }
        ]
      }

Time ranges of indices:

downsample-5m-.ds-my-data-stream-2024.03.27-000002:

"time_series" : {
  "end_time" : "2024-03-27T22:19:05.000Z",
  "start_time" : "2024-03-27T21:34:37.000Z"
},

downsample-5m-.ds-my-data-stream-2024.03.27-000001

"time_series" : {
  "end_time" : "2024-03-27T21:34:37.000Z",
  "start_time" : "2024-03-27T19:04:37.000Z"
},

.ds-my-data-stream-2024.03.27-000003

"time_series" : {
  "end_time" : "2024-03-28T19:14:39.000Z",
  "start_time" : "2024-03-27T22:19:05.000Z"
},

.ds-my-data-stream-2024.03.28-000008

"time_series" : {
  "end_time" : "2024-03-28T19:52:02.000Z",
  "start_time" : "2024-03-28T19:14:39.000Z"
},

We can see that temporal_ranges is wrong. start is 2024-03-27T21:34:37.000Z which is a start time of *000002 index instead of index *000001 with an earlier range. It is likely caused by the order of indices, we probably assume ascending ordering based on time series start and end. It seems like this is not true with downsampling and the root cause is somewhere where we replace backing index of datastream.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[lkts]

This is because of this logic

elasticsearch/modules/data-streams/src/main/java/org/elasticsearch/datastreams/lifecycle/downsampling/DeleteSourceAndAddDownsampleToDS.java

Line 144 in c9b5aa0

private static ClusterState addDownsampleIndexToDataStream(

which calls DataStream#addBackingIndex that does backingIndices.add(0, index);. This is why downsampled indices are always at the top.

[lkts]

ILM works correctly since it uses

elasticsearch/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ilm/DownsampleAction.java

Line 269 in c9b5aa0

ReplaceDataStreamBackingIndexStep replaceDataStreamBackingIndex = new ReplaceDataStreamBackingIndexStep(

which correctly performs DataStream#replaceBackingIndex.