Merges might not be picked up when they are ready #5779
Labels
Comments
s1monw
added a commit
that referenced
this issue
Apr 12, 2014
Due to the default of `async_merge` to `true` we never run the merge policy on a segment flush which prevented the pending merges from being updated and that caused actual pending merges not to contribute to the merge decision. This commit also removes the `index.async.merge` setting is actually misleading since we take care of merges not being excecuted on the indexing threads on a different level (the merge scheduler) since 1.1. This commit also adds an additional check when to run a refresh since soely relying on the dirty flag might leave merges un-refreshed which can cause search slowdowns and higher memory consumption. Closes #5779
s1monw
added a commit
that referenced
this issue
Apr 12, 2014
Due to the default of `async_merge` to `true` we never run the merge policy on a segment flush which prevented the pending merges from being updated and that caused actual pending merges not to contribute to the merge decision. This commit also removes the `index.async.merge` setting is actually misleading since we take care of merges not being excecuted on the indexing threads on a different level (the merge scheduler) since 1.1. This commit also adds an additional check when to run a refresh since soely relying on the dirty flag might leave merges un-refreshed which can cause search slowdowns and higher memory consumption. Closes #5779
mute
pushed a commit
to mute/elasticsearch
that referenced
this issue
Jul 29, 2015
Due to the default of `async_merge` to `true` we never run the merge policy on a segment flush which prevented the pending merges from being updated and that caused actual pending merges not to contribute to the merge decision. This commit also removes the `index.async.merge` setting is actually misleading since we take care of merges not being excecuted on the indexing threads on a different level (the merge scheduler) since 1.1. This commit also adds an additional check when to run a refresh since soely relying on the dirty flag might leave merges un-refreshed which can cause search slowdowns and higher memory consumption. Closes elastic#5779
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We currently play some tricks to make sure merges are not picked up on indexing threads such that the threadpool that is used for indexing is not consumed by merges. Yet, there are a couple of problems that prevent merges to be picked up since we are checking if the IW has pending merges but that datastructure might not be picked up due to a problem in the way we try to prevent the merges to happen. There is also a problem where we don't refresh a reader due to merges where we could / should do that. This does NOT have an impact on correctness but it can have an impact on the amount of segments that are used for searching and for resolving versions to. Both have performance impacts if the number of segments is large.
The text was updated successfully, but these errors were encountered: