New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve filtering for standard aggregation queries. #4056

Merged
merged 2 commits into from Aug 10, 2017

Conversation

Projects
None yet
2 participants
@dennisoelkers
Member

dennisoelkers commented Aug 3, 2017

Description

Motivation and Context

This change improves the queries generated by the Searches#terms/fieldStats/histogram/fieldHistogram methods. By filtering the search result based on the specified time range in the query instead of the aggregation, overall ES runtime is improved.

Fortunately, filteredSearchRequest is already returning a matching query + time range search builder which can be used for those cases. Before this change, the source builder was generated by the specific method and contained only the query instead of query + time range.

Thanks to @hc4 for the input.

Fixes #4051.

How Has This Been Tested?

Screenshots (if appropriate):

Sample runtimes for the histogram before the change:

screen shot 2017-08-03 at 10 52 02

Sample runtimes for the histogram after the change:

screen shot 2017-08-03 at 10 51 32

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

dennisoelkers added some commits Aug 3, 2017

Filter by time range in search query instead of aggregation.
This change improves the queries generated by the
Searches#terms/fieldStats/histogram/fieldHistogram methods. By filtering
the search result based on the specified time range in the query instead
of the aggregation, overall ES runtime is improved.

Fortunately, filteredSearchRequest is already returning a matching query
+ time range search builder which can be used for those cases. Before
this change, the source builder was generated by the specific method and
contained only the query instead of query + time range.

Fixes #4051.
@kroepke

lgtm!

@kroepke kroepke merged commit 0865385 into master Aug 10, 2017

4 checks passed

ci-web-linter Jenkins build graylog-pr-linter-check 1842 has succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
graylog-project/pr Jenkins build graylog-project-pr-snapshot 359 has succeeded
Details

@kroepke kroepke deleted the improve-histogram-query branch Aug 10, 2017

@kroepke kroepke referenced this pull request Aug 10, 2017

Closed

Slow histogram request #4051

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment