Do not alloc full buffer for small change requests #35158
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today we always allocate a full buffer (1024 elements) even though the requesting size is much smaller. With this change, we will use the requesting size as the buffer size if it's smaller than the default batch size; otherwise uses the default batch.
dnhatn
added
the
:Distributed/Engine
|
Pinging @elastic/es-distributed |
martijnvg
reviewed
Nov 2, 2018
| @@ -95,14 +95,15 @@ | |||
| } | |||
| }; | |||
| this.mapperService = mapperService; | |||
| this.searchBatchSize = searchBatchSize; | |||
| final long requestingSize = (toSeqNo - fromSeqNo) == Long.MAX_VALUE ? Long.MAX_VALUE : (toSeqNo - fromSeqNo + 1L); | |||
There was a problem hiding this comment.
(toSeqNo - fromSeqNo) == Long.MAX_VALUE this would practically always evaluate to false?
There was a problem hiding this comment.
Yes for CCR but there is another usage (i.e, peer recovery) where to_seq_no=MAX_VALUE and from_seq_no can be zero:
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/index/engine/InternalEngine.java#L496.
There was a problem hiding this comment.
Got it, thanks for explaining!
|
Thanks everyone :) |
dnhatn
added a commit
that referenced
this pull request
Nov 2, 2018
Today we always allocate a full buffer (1024 elements) in a LuceneChangesSnapshot even though the requesting size is smaller. With this change, we will use the requesting size as the buffer size if it's smaller than the default batch size; otherwise uses the default batch size.
dnhatn
added a commit
that referenced
this pull request
Nov 2, 2018
Today we always allocate a full buffer (1024 elements) in a LuceneChangesSnapshot even though the requesting size is smaller. With this change, we will use the requesting size as the buffer size if it's smaller than the default batch size; otherwise uses the default batch size.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today we always allocate a full buffer (1024 elements) in a LuceneChangesSnapshot even though the requesting size is much smaller. With this change, we will use the requesting size as the buffer size if it's smaller than the default batch size; otherwise uses the default batch size.