Support date math in index names for all index based apis

[martijnvg]

Very often it is useful to query indices within a particular timeframe. For example in the case of weekly indices all the indices of the last month. This can be achieved via aliases, however this does require that something is updating the aliases each time new indices get introduced. Also when the search request (or part of the search request) is stored elsewhere already updating the search request and the alias itself isn't straight forward, because now the search request needs to be maintained in two places.

Instead it would be better if date math is supported in the index names option for all index based apis (search, index, percolate etc.). By adding support for the following format in index names:

<static_name{date_math_expr{optional_date_format}}>

The following index names can be specified:

• <logstash-{now/d}> translates into logstash-2024.03.22
• <logstash-{now/M}> translates into logstash-2024.03.01
• <logstash-{now/M{YYYY.MM}}> translates into logstash-2024.03
• <logstash-{now/M-1M{YYYY.MM}}> translates into logstash-2024.02

The data math expressions should be resolved before wildcard expressions are resolved during the concrete index name resolution.

Before this feature can be added the code that currently exists for index name wildcard expansions, index to concrete index resolutions should be refactored first to support this nicely without hacks. See #12058.

[nik9000]

Very cute.

[ricardojohnny]

Can anyone tell me how to proceed or the name of an index is like:

"logstash-september-000009"
"logstash-october-000010"
"logstash-november-000011"

This applies to the "provided_name" of the monthly index lifecycle.

[gwbrown]

@ricardojohnny This appears to be a user question, and we'd like to direct these kinds of things to the forums. If you can stop by there, we'd appreciate it. This allows us to use GitHub for verified bug reports, feature requests, and pull requests.

There's an active community in the forums that should be able to help get an answer to your question.