New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TieredMergePolicy.findMerges improvements #219

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
1 participant
@outcoldman
Copy link

outcoldman commented Jul 13, 2017

Calculate the total size of index from the beginning by excluding too
big segments.
When finding all the merges take in account that merges also produces
segments.

Just something I have found while looking on the code and understanding how default elasticsearch settings applies to lucese merging strategy.
Not sure if that worth the effort of taking in master.
I have not tested it and have not even tried to compile.

TieredMergePolicy.findMerges improvements
Calculate the total size of index from the beginning by excluding too
big segments.
When finding all the merges take in account that merges also produces
segments.
@outcoldman

This comment has been minimized.

Copy link

outcoldman commented Jul 13, 2017

Also looking a little bit more on the code. As I understand segments with size larger than maxMergedSegmentBytes/2.0 never will be included in any merges, even if amount of deleted documents will be more than forceMergeDeletesPctAllowed. And the only option to reclaim the space is to force the merges. (using force merge api from Elasticsearch).
Is that expected or a bug?

@outcoldman

This comment has been minimized.

Copy link

outcoldman commented Jul 13, 2017

Also, I am right that if you will end up with segment of size (maxMergedSegmentBytes/2.0 - 1) so it is basically a high chance, that this segment will never be merged, because there are will be no segments with size of 1 byte. Considering that this segment does not have any deletes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment