Make watch history indices hidden #52962
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit updates the template used for watch history indices with hidden index setting so that new indices will be created as hidden. Additionally, some failing tests were encountered where a search would find the documents in the history index but a subsequent search would fail to find the documents. This is most likely due to different refresh times between the primary and replica shards. The failures were resolved by using an assertsBusy to retrieve the documents. Relates elastic#50251
|
Pinging @elastic/es-core-features (:Core/Features/Watcher) |
|
I was confused why all the |
|
Thanks for catching that. I reverted all of those changes |
jaymode
added a commit
to jaymode/elasticsearch
that referenced
this pull request
Feb 28, 2020
This commit updates the template used for watch history indices with the hidden index setting so that new indices will be created as hidden. Relates elastic#50251
jaymode
added a commit
that referenced
this pull request
Mar 6, 2020
jakelandis
added a commit
to jakelandis/elasticsearch
that referenced
this pull request
May 14, 2020
[WatcherIndexTemplateRegistry](https://github.com/elastic/elasticsearch/blob/7.7/x-pack/plugin/watcher/src/main/java/org/elasticsearch/xpack/watcher/support/WatcherIndexTemplateRegistry.java#L74) as of elastic#52962 requires all nodes to be on 7.7.0 before it allows the version 11 index template to be installed. While in a mixed cluster, nothing prevents Watcher from running on the new host before the all of the nodes are on 7.7.0. This will result in the .watcher-history-11* index without the proper mappings. Without the proper mapping a single document (for a large watch) can exceed the default 1000 field limit and cause error to show in the logs. This commit ensures the same logic for writing to the index is applied as for installing the template. In a mixed cluster, the `10` index template will continue to be written. Only once all of nodes are on 7.7.0+ will the `11` index template be installed and used. Note - this PR targets 7.x and will be back ported to 7.7.next. This conditional installation of the template is not present in master. closes elastic#56732
jakelandis
added a commit
that referenced
this pull request
May 15, 2020
#56734) WatcherIndexTemplateRegistry as of #52962 requires all nodes to be on 7.7.0 before it allows the version 11 index template to be installed. While in a mixed cluster, nothing prevents Watcher from running on the new host before the all of the nodes are on 7.7.0. This will result in the .watcher-history-11* index without the proper mappings. Without the proper mapping a single document (for a large watch) can exceed the default 1000 field limit and cause error to show in the logs. This commit ensures the same logic for writing to the index is applied as for installing the template. In a mixed cluster, the `10` index template will continue to be written. Only once all of nodes are on 7.7.0+ will the `11` index template be installed and used. closes #56732
jakelandis
added a commit
to jakelandis/elasticsearch
that referenced
this pull request
May 15, 2020
elastic#56734) WatcherIndexTemplateRegistry as of elastic#52962 requires all nodes to be on 7.7.0 before it allows the version 11 index template to be installed. While in a mixed cluster, nothing prevents Watcher from running on the new host before the all of the nodes are on 7.7.0. This will result in the .watcher-history-11* index without the proper mappings. Without the proper mapping a single document (for a large watch) can exceed the default 1000 field limit and cause error to show in the logs. This commit ensures the same logic for writing to the index is applied as for installing the template. In a mixed cluster, the `10` index template will continue to be written. Only once all of nodes are on 7.7.0+ will the `11` index template be installed and used. closes elastic#56732
jakelandis
added a commit
that referenced
this pull request
May 15, 2020
… to use (#56734) (#56848) WatcherIndexTemplateRegistry as of #52962 requires all nodes to be on 7.7.0 before it allows the version 11 index template to be installed. While in a mixed cluster, nothing prevents Watcher from running on the new host before the all of the nodes are on 7.7.0. This will result in the .watcher-history-11* index without the proper mappings. Without the proper mapping a single document (for a large watch) can exceed the default 1000 field limit and cause error to show in the logs. This commit ensures the same logic for writing to the index is applied as for installing the template. In a mixed cluster, the `10` index template will continue to be written. Only once all of nodes are on 7.7.0+ will the `11` index template be installed and used. closes #56732
jakelandis
added a commit
to jakelandis/elasticsearch
that referenced
this pull request
May 15, 2020
elastic#56734) WatcherIndexTemplateRegistry as of elastic#52962 requires all nodes to be on 7.7.0 before it allows the version 11 index template to be installed. While in a mixed cluster, nothing prevents Watcher from running on the new host before the all of the nodes are on 7.7.0. This will result in the .watcher-history-11* index without the proper mappings. Without the proper mapping a single document (for a large watch) can exceed the default 1000 field limit and cause error to show in the logs. This commit ensures the same logic for writing to the index is applied as for installing the template. In a mixed cluster, the `10` index template will continue to be written. Only once all of nodes are on 7.7.0+ will the `11` index template be installed and used. closes elastic#56732
jakelandis
added a commit
that referenced
this pull request
May 26, 2020
… to use (#56734) (#56855) WatcherIndexTemplateRegistry as of #52962 requires all nodes to be on 7.7.0 before it allows the version 11 index template to be installed. While in a mixed cluster, nothing prevents Watcher from running on the new host before the all of the nodes are on 7.7.0. This will result in the .watcher-history-11* index without the proper mappings. Without the proper mapping a single document (for a large watch) can exceed the default 1000 field limit and cause error to show in the logs. This commit ensures the same logic for writing to the index is applied as for installing the template. In a mixed cluster, the `10` index template will continue to be written. Only once all of nodes are on 7.7.0+ will the `11` index template be installed and used. closes #56732
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit updates the template used for watch history indices with
hidden index setting so that new indices will be created as hidden.
Relates #50251