-
Notifications
You must be signed in to change notification settings - Fork 24.6k
-
Notifications
You must be signed in to change notification settings - Fork 24.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Internal: Optimize with force=true
can sidestep max segments to merge at once used by delegate
#7904
Labels
Comments
This is quite nasty: if a shard has many segments, this can suck up lots of RAM, file descriptors, take much longer to run than if we let the merge policy do separate merges ... I wonder how often users are "forcing" their optimize. |
rjernst
added a commit
to rjernst/elasticsearch
that referenced
this issue
Sep 29, 2014
This does the following: * Make 'force' flag only build a merge if the delegate MP returned no merges * Add async handling for 'flush' when 'waitForMerges' is false * Remove flush at the beginning of optimize. This is something the user can do if they wish, before calling optimize. closes elastic#7886 closes elastic#7904
rjernst
added a commit
that referenced
this issue
Sep 29, 2014
This does the following: * Make 'force' flag only build a merge if the delegate MP returned no merges * Add async handling for 'flush' when 'waitForMerges' is false * Remove flush at the beginning of optimize. This is something the user can do if they wish, before calling optimize. closes #7886 closes #7904 closes #7920
rjernst
added a commit
that referenced
this issue
Sep 30, 2014
This does the following: * Make 'force' flag only build a merge if the delegate MP returned no merges * Add async handling for 'flush' when 'waitForMerges' is false * Remove flush at the beginning of optimize. This is something the user can do if they wish, before calling optimize. closes #7886 closes #7904 closes #7920
clintongormley
changed the title
Optimize with
Internal: Optimize with Oct 1, 2014
force=true
can sidestep max segments to merge at once used by delegateforce=true
can sidestep max segments to merge at once used by delegate
mute
pushed a commit
to mute/elasticsearch
that referenced
this issue
Jul 29, 2015
This does the following: * Make 'force' flag only build a merge if the delegate MP returned no merges * Add async handling for 'flush' when 'waitForMerges' is false * Remove flush at the beginning of optimize. This is something the user can do if they wish, before calling optimize. closes elastic#7886 closes elastic#7904 closes elastic#7920
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The implementation of "forcing" optimize right now will cause the ElasticsearchMergePolicy to return one giant OneMerge if
force==true
. However, this can cause IO issues. The existing MP impls chain merging through "cascading", so that no OneMerge merges more than some X segments (e.g. X = 30 for TieredMP). Forcing should do the same...The text was updated successfully, but these errors were encountered: