-
Notifications
You must be signed in to change notification settings - Fork 24.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dangling indices import ignores aliases #8059
Conversation
Dangling indices are indexes found on disk which are not part of the cluster state. By default, we don't delete them but rather import them into the cluster state in order to not accidentally delete data and also allow for the ease of copying index data folders from one cluster to another. Currently, the import logic doesn't check for existing aliases of the same name as the imported dangling index, causing both an index and an alias with the same name. This commit add a protection against this. Note that the index is still kept as dangling and is only deleted from disk after `gateway.local.dangling_timeout` has passed (2 hours). We also change the log message indicating deletion of dangling indices to a `WARN` level.
if (currentState.metaData().aliases().containsKey(indexMetaData.index())) { | ||
logger.warn("ignoring dangled index [{}] on node [{}] due to an existing alias with the same name", | ||
indexMetaData.index(), request.fromNode); | ||
continue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should remove the continue
here it's dangerous to not import it - rather live with the name clash until it's resolved?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bleskes has the concern that this state is not defined in our system (since we protect on it from the "outside"), and the implications of having this clash is not evident. I was leaning towards living with the clash, but since its not defined, this change might be good enough for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok good - better than an undefined state!
LGTM |
Dangling indices are indexes found on disk which are not part of the cluster state. By default, we don't delete them but rather import them into the cluster state in order to not accidentally delete data and also allow for the ease of copying index data folders from one cluster to another. Currently, the import logic doesn't check for existing aliases of the same name as the imported dangling index, causing both an index and an alias with the same name. This commit add a protection against this. Note that the index is still kept as dangling and is only deleted from disk after `gateway.local.dangling_timeout` has passed (2 hours). We also change the log message indicating deletion of dangling indices to a `WARN` level. Closes #8059
Dangling indices are indexes found on disk which are not part of the cluster state. By default, we don't delete them but rather import them into the cluster state in order to not accidentally delete data and also allow for the ease of copying index data folders from one cluster to another. Currently, the import logic doesn't check for existing aliases of the same name as the imported dangling index, causing both an index and an alias with the same name.
This commit add a protection against this. Note that the index is still kept as dangling and is only deleted from disk after
gateway.local.dangling_timeout
has passed (2 hours).We also change the log message indicating deletion of dangling indices to a
WARN
level.