Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ExitableDirectoryReader slowdowns intensive points and terms query #53496

Closed
jimczi opened this issue Mar 12, 2020 · 1 comment · Fixed by #53712
Closed

ExitableDirectoryReader slowdowns intensive points and terms query #53496

jimczi opened this issue Mar 12, 2020 · 1 comment · Fixed by #53712
Assignees
@matriv
Labels
blocker >regression :Search/Search Search-related issues that do not fall into other categories v7.7.0

Comments

@jimczi
Copy link
Contributor

jimczi commented Mar 12, 2020

The exitable directory reader added in #52822 seems to slowdown queries that uses points (range queries) and terms dictionary (terms ad multi-term queries) intensively. These slowdowns have been spotted in our nightly runs as outlined here.
We open this issue to discuss the best ways to mitigate these slowdowns. The first thing we'd like to try is to make the interval at which we check for cancellation or timeout smaller since we started with a relatively low value (15).
@jimczi jimczi added >regression blocker :Search/Search Search-related issues that do not fall into other categories v7.7.0 labels Mar 12, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@markharwood markharwood mentioned this issue Mar 13, 2020
Closed
matriv added a commit to matriv/elasticsearch that referenced this issue Mar 18, 2020
@matriv
Increase step between checks for cancellation
a1839d0 
Check for cancellation every 1024 docs instead of every 15 to lower
the impact of the check in query's performance.

Fixes: elastic#53496
@matriv matriv mentioned this issue Mar 18, 2020
Merged
matriv added a commit that referenced this issue Mar 18, 2020
@matriv
Increase step between checks for cancellation (#53712)
6b5fc35 
The introduction of the ExitableDirectoryReader showed increase of
latencies for range queries using pointvalues.

Check for cancellation every 1024 docs instead of every 15 to lower
the impact of the check in query's performance.

Follows: #52822
Fixes: #53496
matriv added a commit that referenced this issue Mar 18, 2020
@matriv
Increase step between checks for cancellation (#53712)
d56dee5 
The introduction of the ExitableDirectoryReader showed increase of
latencies for range queries using pointvalues.

Check for cancellation every 1024 docs instead of every 15 to lower
the impact of the check in query's performance.

Follows: #52822
Fixes: #53496
(cherry picked from commit 6b5fc35)
matriv added a commit to matriv/elasticsearch that referenced this issue Mar 23, 2020
@matriv
Reduce performance impact of ExitableDirectoryReader
8f23bc7 
Benchmarking showed that the effect of the ExitableDirectoryReader
is reduced considerably when checking every 4095 docs. Moreover,
set the cancellable task before calling QueryPhase.preProcess()
and make sure we don't wrap with an ExitableDirectoryReader at all
when lowLevelCancellation() is set to false to avoid completely any
performance impact.

Follows: elastic#52822
Follows: elastic#53166
Follows: elastic#53496
@matriv matriv mentioned this issue Mar 23, 2020
Merged
matriv added a commit that referenced this issue Mar 23, 2020
@matriv
Reduce performance impact of ExitableDirectoryReader (#53978)
cdc377e 
Benchmarking showed that the effect of the ExitableDirectoryReader
is reduced considerably when checking every 8191 docs. Moreover,
set the cancellable task before calling QueryPhase#preProcess()
and make sure we don't wrap with an ExitableDirectoryReader at all
when lowLevelCancellation is set to false to avoid completely any
performance impact.

Follows: #52822
Follows: #53166
Follows: #53496
@matriv matriv mentioned this issue Mar 23, 2020
Merged
matriv added a commit that referenced this issue Mar 23, 2020
@matriv
Reduce performance impact of ExitableDirectoryReader (#53978) (#54014)
3a3e964 
Benchmarking showed that the effect of the ExitableDirectoryReader
is reduced considerably when checking every 8191 docs. Moreover,
set the cancellable task before calling QueryPhase#preProcess()
and make sure we don't wrap with an ExitableDirectoryReader at all
when lowLevelCancellation is set to false to avoid completely any
performance impact.

Follows: #52822
Follows: #53166
Follows: #53496

(cherry picked from commit cdc377e)
@codebrain codebrain mentioned this issue Apr 1, 2020
Closed
38 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matriv matriv

Labels
blocker >regression :Search/Search Search-related issues that do not fall into other categories v7.7.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Increase step between checks for cancellation matriv/elasticsearch
3 participants
@matriv @elasticmachine @jimczi