Make mapping updates atomic wrt document parsing. #11205
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When mapping updates happen concurrently with document parsing, bad things can happen. For instance, when applying a mapping update we first update the Mapping object which is used for parsing and then FieldNameAnalyzer which is used by IndexWriter for analysis. So if you are unlucky, it could happen that a document was parsed successfully without introducing dynamic updates yet IndexWriter does not see its analyzer yet. In order to fix this issue, mapping updates are now protected by a write lock and document parsing is protected by the read lock associated with this write lock. This ensures that no documents will be parsed while a mapping update is being applied, so document parsing will either see none of the update or all of it.
jpountz
added
v2.0.0-beta1
:Search/Mapping
|
LGTM. Any way to test this? |
|
Merged manually. |
clintongormley
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When mapping updates happen concurrently with document parsing, bad things can
happen. For instance, when applying a mapping update we first update the Mapping
object which is used for parsing and then FieldNameAnalyzer which is used by
IndexWriter for analysis. So if you are unlucky, it could happen that a document
was parsed successfully without introducing dynamic updates yet IndexWriter does
not see its analyzer yet.
In order to fix this issue, mapping updates are now protected by a write lock
and document parsing is protected by the read lock associated with this write
lock. This ensures that no documents will be parsed while a mapping update is
being applied, so document parsing will either see none of the update or all of
it.