During relocation, process pending mapping update in phase 2 #6762
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
During phase1 we copy over all lucene segments. These make refer to mapping updates that are still queued up to be sent to master. We must make sure those pending updates are sent before completing the relocation. Relates to elastic#6648
|
I think that there is a cleaner way to do it, by simply sending an update mapping per type, with a listener, and waiting for the response. This will guarantee that all the mappings have been properly processed by the master node. |
…changes not only pending
|
@kimchy pushed another commit based on your suggestion |
| @@ -251,22 +258,55 @@ public void phase2(Translog.Snapshot snapshot) throws ElasticsearchException { | |||
| if (shard.state() == IndexShardState.CLOSED) { | |||
| throw new IndexShardClosedException(request.shardId()); | |||
| } | |||
| logger.trace("[{}][{}] recovery [phase2] to {}: start", request.shardId().index().name(), request.shardId().id(), request.targetNode()); | |||
| logger.trace("{} recovery [phase2] to {}: start", request.shardId().index().name(), request.shardId().id(), request.targetNode()); | |||
There was a problem hiding this comment.
why the change to the logging? now we don't properly log the parameters?
There was a problem hiding this comment.
I think you forgot to change it to request.shardId as a single parameter, instead of the index name and shard id both
|
left minor comment, LGTM otherwise |
clintongormley
added a commit
that referenced
this pull request
Jul 7, 2014
Delete and update mapping execution order has changed with #6762
clintongormley
added a commit
that referenced
this pull request
Jul 7, 2014
Delete and update mapping execution order has changed with #6762
bleskes
added a commit
to bleskes/elasticsearch
that referenced
this pull request
Jul 7, 2014
The change added in elastic#6762 helps making sure the pending mapping updates are processed on all nodes to prevent moving shards to nodes which are not yet fully aware of the new mapping. However it introduced a racing condition delete_mapping operations, potentially causing a type to be added after it's deletion. This commit solves this by only sending a mapping update if the mapping source has actually changed.
bleskes
added a commit
that referenced
this pull request
Jul 7, 2014
The change added in #6762 helps making sure the pending mapping updates are processed on all nodes to prevent moving shards to nodes which are not yet fully aware of the new mapping. However it introduced a racing condition delete_mapping operations, potentially causing a type to be added after it's deletion. This commit solves this by only sending a mapping update if the mapping source has actually changed. Closes #6772
bleskes
added a commit
that referenced
this pull request
Jul 7, 2014
The change added in #6762 helps making sure the pending mapping updates are processed on all nodes to prevent moving shards to nodes which are not yet fully aware of the new mapping. However it introduced a racing condition delete_mapping operations, potentially causing a type to be added after it's deletion. This commit solves this by only sending a mapping update if the mapping source has actually changed. Closes #6772
clintongormley
changed the title
clintongormley
changed the title
clintongormley
added
:Distributed/Distributed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
During phase1 we copy over all lucene segments. These make refer to mapping updates that are still queued up to be sent to master. We must make sure those pending updates are sent before completing the relocation.
Relates to #6648