Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate index format agreement for system index descriptors #85173

Merged

Conversation

williamrandolph
Copy link
Contributor

When the SystemIndexManager class checks the index format for an internally managed system index, it compares a value in the system index descriptor with a value in the index's stored settings. If the two values differ, SystemIndexManager concludes that some external migration needs to be triggered before it can update the system index's mappings and metadata. However, if our code had different numbers in the descriptor and in the settings, that descriptor's system indices will always be in a state where they can't be updated, since users can't override the defined settings.

This PR adds validation to the system index descriptor so that we will catch any system index descriptors with this problem, and we add bwc tests for the watcher case.

Relates #84880

@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Mar 21, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine
Copy link
Collaborator

Hi @williamrandolph, I've created a changelog YAML for you.

@williamrandolph williamrandolph added the test-full-bwc Trigger full BWC version matrix tests label Mar 21, 2022
// post a watch
Request putWatchRequest = new Request("PUT", "_watcher/watch/log_error_watch");
putWatchRequest.setJsonEntity(
"{\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use multiline strings here now

@williamrandolph williamrandolph added the auto-backport-and-merge Automatically create backport pull requests and merge when ready label Mar 22, 2022
@williamrandolph williamrandolph merged commit 1307a98 into elastic:master Mar 22, 2022
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
7.17 Commit could not be cherrypicked due to conflicts
8.1 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 85173

williamrandolph added a commit to williamrandolph/elasticsearch that referenced this pull request Mar 23, 2022
…#85173)

* Validate index format for system indices
* Add bwc test for watcher meta version issue
* Update docs/changelog/85173.yaml
williamrandolph added a commit to williamrandolph/elasticsearch that referenced this pull request Mar 23, 2022
…#85173)

* Validate index format for system indices
* Add bwc test for watcher meta version issue
* Update docs/changelog/85173.yaml
@williamrandolph
Copy link
Contributor Author

Backports:

7.17: #85290
8.1: #85289

elasticsearchmachine pushed a commit that referenced this pull request Mar 23, 2022
…#85289)

* Validate index format for system indices
* Add bwc test for watcher meta version issue
* Update docs/changelog/85173.yaml
elasticsearchmachine pushed a commit that referenced this pull request Mar 23, 2022
…#85290)

* Validate index format agreement for system index descriptors (#85173)

* Validate index format for system indices
* Add bwc test for watcher meta version issue
* Update docs/changelog/85173.yaml

* Add include_type_name warning handler for bwc tests
@williamrandolph williamrandolph deleted the validate-system-index-format branch May 23, 2022 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport-and-merge Automatically create backport pull requests and merge when ready >bug :Core/Infra/Core Core issues without another label Team:Core/Infra Meta label for core/infra team test-full-bwc Trigger full BWC version matrix tests v7.17.2 v8.1.2 v8.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants