[Downsampling] Enhance telemetry. #133953
Description
Description
We would like to get a better insights in how downsampling is currently used. In this initial effort we would like to answer the following questions:
- What percentage of TSDS data streams have at least one downsampling round configured?
- How many managed by ILM and how many by DLM?
- How many are the average, min, max rounds split by ILM & DLM?
- How many data streams have an interval configured?
- How many downsampled indices do we have per interval?
This could like:
"time_series": {
"available": true,
"enabled": true,
"data_stream_count": 10,
"index_count": 100,
"downsampling": {
"index_count_per_interval": {
"5m": 5,
"10m": 10,
"1h": 10000
},
"ilm": {
"data_stream_count": 8,
"index_count": 50,
"rounds_per_data_stream": {
"min": 1,
"max": 3,
"average": 2
},
"phases_in_use": {
"hot": 10,
"warm": 5,
"cold": 10
}
},
"dlm": {
"downsampled_data_stream_count": 8,
"downsampled_index_count": 50,
"rounds_per_data_stream": {
"min": 1,
"max": 3,
"average": 2
}
}
}
}
When we have no time series data streams the following response is sufficient considering that we only count time series data streams:
"time_series": {
"available": true,
"enabled": true,
"data_stream_count": 0
}
And when we have time series data streams but no downsampled data streams:
"time_series": {
"available": true,
"enabled": true,
"data_stream_count": 10,
"index_count": 100,
"downsampling": {
"ilm": {
"data_stream_count": 0
},
"dlm": {
"downsampled_data_stream_count": 0
}
}
}
This way we can also track the clusters with time series data streams and no downsampling configured pretty easily.