Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use `SEARCH` threadpool for potentially blocking operations #7624

Merged
merged 1 commit into from Sep 8, 2014

Conversation

Projects
None yet
3 participants
@s1monw
Copy link
Contributor

s1monw commented Sep 6, 2014

In SearchServiceTransportAction we use SAME threadpool for
potentially blocking / longer running operations this can cause
slowdowns in response / requests processing or even deadlocks. This
commit uses SEARCH threadpool for all operations except of freeing
contexts.

Closes #7623

@s1monw s1monw added the review label Sep 6, 2014

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Sep 6, 2014

this change would be a shame to do, since if you execute a 10 shard search, the first 9 will just spawn a thread to add the response to the datastrucutre and return, not really execute anything.

I think the actual second / third phase of execution, mainly the reduce of the second query phase, where aggs are being reduced, should be done on the SEARCH thread pool.

@s1monw

This comment has been minimized.

Copy link
Contributor Author

s1monw commented Sep 6, 2014

@kimchy this makes no sense indeed. the change was pretty naive too though... Yet, I tried to only execute the reduce phases / finish phases in a search thread but this was trickier that I thought. I pushed a working patch with a big nocommit - would love to discuss

@s1monw s1monw force-pushed the s1monw:issues/7623 branch Sep 8, 2014

@s1monw

This comment has been minimized.

Copy link
Contributor Author

s1monw commented Sep 8, 2014

I removed the pending no-commits I think it's ready

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Sep 8, 2014

lgtm

[SEARCH] Execute search reduce phase on the search threadpool
Reduce Phases can be expensive and some of them like the aggregations
reduce phase might even execute a one-off call via an internal client
that might cause a deadlock due to execution on the network thread
that is needed to handle the one-off call. This commit dispatches
the reduce phase to the search threadpool to ensure we don't wait
for the current thread to be available.

Closes #7623

@s1monw s1monw force-pushed the s1monw:issues/7623 branch to aadbfa4 Sep 8, 2014

@s1monw s1monw removed the review label Sep 8, 2014

@s1monw s1monw merged commit aadbfa4 into elastic:master Sep 8, 2014

@s1monw s1monw deleted the s1monw:issues/7623 branch Sep 8, 2014

@clintongormley clintongormley changed the title [SEARCH] Use `SEARCH` threadpool for potentially blocking operations Internal: Use `SEARCH` threadpool for potentially blocking operations Sep 8, 2014

@clintongormley clintongormley changed the title Internal: Use `SEARCH` threadpool for potentially blocking operations Use `SEARCH` threadpool for potentially blocking operations Jun 7, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.