Add date math support in index names #12209
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
martijnvg
added
>feature
v2.0.0-beta1
review
:Core/Infra/Core
martijnvg
force-pushed
the
index_name_date_math
branch
from
01b7e84
to
754a106
Compare
|
|
||
| Where: | ||
|
|
||
| * `static_name` is the static text part of the name |
There was a problem hiding this comment.
format this as:
[horizontal]
`static_name`:: The static text part of the name
`date_math_expr`:: A dynamic....
There was a problem hiding this comment.
Looks much nicer now :)
|
Updated the pr to fix @clintongormley's docs feedback. |
| import static org.hamcrest.Matchers.is; | ||
| import static org.hamcrest.Matchers.notNullValue; | ||
|
|
||
| @ElasticsearchIntegrationTest.ClusterScope(numDataNodes = 1) |
There was a problem hiding this comment.
I did this just to make the the test run a bit faster during development. This can be removed
|
left a few comments |
martijnvg
force-pushed
the
index_name_date_math
branch
from
232a28c
to
1892452
Compare
|
@javanna I rebased the PR to master and applied your comments. |
| // TODO: I think startTime() should become part of ActionRequest and that should be used both for index name | ||
| // data math expressions and $now in scripts. This way all apis will deal with now in the same way instead | ||
| // of just for the _search api | ||
| String[] concreteIndices = indexNameExpressionResolver.concreteIndices(clusterState, request.indicesOptions(), startTime(), request.indices()); |
There was a problem hiding this comment.
this //TODO will have to stay right? maybe we want to open an issue instead to keep track of this? also s/data/date
|
did another round, left a couple minor comments |
|
@javanna I updated the PR. |
| * @return whether the specified alias or index exists. If the alias or index contains datemath then that is resolved too. | ||
| */ | ||
| public boolean hasIndexOrAliasAndResolveDateMath(String aliasOrIndex, ClusterState state) { | ||
| assert expressionResolvers.get(0) instanceof DateMathExpressionResolver; |
There was a problem hiding this comment.
given that we add the expression resolvers manually to the list, maybe keep a reference to it instead so we can use it directly instead of getting it from the list?
|
LGTM |
Date math index name resolution enables you to search a range of time-series indices, rather than searching all of your time-series indices and filtering the the results or maintaining aliases. Limiting the number of indices that are searched reduces the load on the cluster and improves execution performance. For example, if you are searching for errors in your daily logs, you can use a date math name template to restrict the search to the past two days.
The added `ExpressionResolver` implementation that is responsible for resolving date math expressions in index names. This resolver is evaluated before wildcard expressions are evaluated.
The supported format: `<static_name{date_math_expr{date_format|timezone_id}}>` and the date math expressions must be enclosed within angle brackets. The `date_format` is optional and defaults to `YYYY.MM.dd`. The `timezone_id` id is optional too and defaults to `utc`.
The `{` character can be escaped by places `\\` before it.
Closes elastic#12059
martijnvg
force-pushed
the
index_name_date_math
branch
from
091e686
to
ac3d090
Compare
clintongormley
added
release highlight
:Data Management/Indices APIs
This was referenced Nov 21, 2015
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Date math index name resolution enables you to search a range of time-series indices, rather than searching all of your time-series indices and filtering the the results or maintaining aliases. Limiting the number of indices that are searched reduces the load on the cluster and improves execution performance. For example, if you are searching for errors in your daily logs, you can use a date math name template to restrict the search to the past two days.
The added
ExpressionResolverimplementation that is responsible for resolving date math expressions in index names. This resolver is evaluated before wildcard expressions are evaluated.The supported format:
<static_name{date_math_expr{date_format|timezone_id}}>and the date math expressions must be enclosed within angle brackets. Thedate_formatis optional and defaults toYYYY.MM.dd. Thetimezone_idid is optional too and defaults toutc.The
{character can be escaped by places\\before it.The following index names can be specified: (assuming now is 2024.03.22 noon utc)
<logstash-{now/d}>translates intologstash-2024.03.22<logstash-{now/M}>translates intologstash-2024.03.01<logstash-{now/M{YYYY.MM}}>translates intologstash-2024.03<logstash-{now/M-1M{YYYY.MM}}>translates intologstash-2024.02<logstash-{now/d{YYYY.MM.dd|+12:00}}>translates intologstash-2024.03.23The following example shows a search request that searches the Logstash indices for the past
three days:
PR for #12059