New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Also throttle delete by query when merges fall behind #9986

Closed
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@mikemccand
Contributor

mikemccand commented Mar 4, 2015

Delete by query is extremely costly: it forces a refresh every time, which can create many segments if there is also concurrent indexing. This changes throttles delete by query so only 1 thread can run at a time if merges are falling behind.

@rjernst

This comment has been minimized.

Member

rjernst commented Mar 4, 2015

LGTM

@mikemccand mikemccand closed this in 24d8c74 Mar 4, 2015

mikemccand added a commit that referenced this pull request Mar 4, 2015

Core: throttle delete-by-query when merges are falling behind
Delete-by-query is incredibly costly because it forces a refresh each
time, so if you are also indexing this can cause massive segment
explosion.

This change throttles delete-by-query when merges can't keep up.  It's
likely not enough (#7052 is the long-term solution) but can only
help.

Closes #9986

mikemccand added a commit that referenced this pull request Mar 4, 2015

Core: throttle delete-by-query when merges are falling behind
Delete-by-query is incredibly costly because it forces a refresh each
time, so if you are also indexing this can cause massive segment
explosion.

This change throttles delete-by-query when merges can't keep up.  It's
likely not enough (#7052 is the long-term solution) but can only
help.

Closes #9986

@clintongormley clintongormley changed the title from Core: also throttle delete by query when merges fall behind to Also throttle delete by query when merges fall behind Jun 7, 2015

mute pushed a commit to mute/elasticsearch that referenced this pull request Jul 29, 2015

Core: throttle delete-by-query when merges are falling behind
Delete-by-query is incredibly costly because it forces a refresh each
time, so if you are also indexing this can cause massive segment
explosion.

This change throttles delete-by-query when merges can't keep up.  It's
likely not enough (elastic#7052 is the long-term solution) but can only
help.

Closes elastic#9986
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment