Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve reduction of terms aggregations #61779

Merged
merged 17 commits into from
Sep 4, 2020
Merged

Improve reduction of terms aggregations #61779

merged 17 commits into from
Sep 4, 2020

Commits on Sep 1, 2020

  1. Improve reduction of terms aggregations 

    Today, the terms aggregation reduces multiple aggregations at once using a map
to group same buckets together. This operation can be costly since it requires
to lookup every bucket in a global map with no particular order.
This commit changes how term buckets are sorted by shards and partial reduces in
order to be able to reduce results using a merge-sort strategy.
For bwc, results are merged with the legacy code if any of the aggregations use
a different sort (if it was returned by a node in prior versions).

Relates elastic#51857
    @jimczi
    jimczi committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    c97a1ed View commit details
    Browse the repository at this point in the history

  2. address feedback

    @jimczi
    jimczi committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    99bc538 View commit details
    Browse the repository at this point in the history

  3. fix the computation of doc_count_error

    @jimczi
    jimczi committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    3a45a61 View commit details
    Browse the repository at this point in the history

  4. handle empty buckets

    @jimczi
    jimczi committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    f060ccd View commit details
    Browse the repository at this point in the history

  5. fix serialization test

    @jimczi
    jimczi committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    b027d81 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2020

  1. fix NPE

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    163ff53 View commit details
    Browse the repository at this point in the history

  2. remove version randomization in query search result tests

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    d8f3a5d View commit details
    Browse the repository at this point in the history

  3. we can prune the list on partial reduce if the aggregation is ordered… 

    … by key and not filtered (minDocCount == 0)
    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    646cfb6 View commit details
    Browse the repository at this point in the history

  4. fix anither it

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    7af5dfb View commit details
    Browse the repository at this point in the history

  5. unused import

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    038700d View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into terms_aggs_merge_reduce

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    a195805 View commit details
    Browse the repository at this point in the history

  7. generalize IteratorAndCurrent with histograms and terms

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    d752ac7 View commit details
    Browse the repository at this point in the history

  8. fix assert

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    902d2c9 View commit details
    Browse the repository at this point in the history

  9. missing next

    @jimczi
    jimczi committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    72d2ded View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2020

  1. set the right comparator in assert

    @jimczi
    jimczi committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    237c686 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into terms_aggs_merge_reduce

    @jimczi
    jimczi committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    ae5d7a8 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2020

  1. Merge branch 'master' into terms_aggs_merge_reduce

    @jimczi
    jimczi committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    2d96c64 View commit details
    Browse the repository at this point in the history