Fixes bug that causes out-of-order sstable key. #2445
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The previous way to address the problem was to replace \u{0000} with 0 in different places.
This logic had several flaws:
Done on the serializer side (like it was for the columnar), there was a collision problem.
If a document in the segment contained a json field with a \0 and antoher doc contained the same json field but
0
then we were sending the same field path twice to the serializer.Another option would have been to normalizes all values on the writer side.
This PR simplifies the logic and simply ignore json path containing a \0, both in the columnar and the inverted index.
Closes #2442