Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Min and Max BKD optimizations :) #44290

Closed
polyfractal opened this issue Jul 12, 2019 · 5 comments
Closed

Optimize Min and Max BKD optimizations :) #44290

polyfractal opened this issue Jul 12, 2019 · 5 comments

Comments

@polyfractal
Copy link
Contributor

The Max aggregator has an optimization to use the BKD tree in an attempt to find the max, bypassing an expensive collection of all documents. It does this by checking the largest leaf in the tree to see if we can find the max. Today this process decodes the packed value for every live doc in the leaf, which is not necessary. We could instead just cache the packed value and decode after intersecting.

The Min aggregator works a little differently. Since values are sorted ascending in the BKD tree, we can start at the beginning and iterate until we find a live doc (e.g. non-deleted document) then exit and use that value.

This is potentially problematic if there are many or mostly deleted documents, since we could spend a long time traversing the BKD tree. It might be faster to actually collect the documents normally since those skip deleted documents. We should probably include some kind of heuristic and revert to the non-BKD approach if we can't find the value (max 1024 lookups?)

This would be a good first issue for someone wanting to get into the agg framework, or learn how the BKD tree works, or both :)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo

@michalperlak
Copy link
Contributor

Hi @polyfractal,
Can I work on this issue?

@polyfractal
Copy link
Contributor Author

@michalperlak Absolutely! Let me know if you have any questions :)

@deXetrous
Copy link

Hi @polyfractal I want to work on this if that's fine. Can you please help me to get started and understand the code base?

@polyfractal
Copy link
Contributor Author

Oops, this ticket should be closed actually. It was implemented by #44315 linked above.

Sorry! There are other issues labeled with low hanging fruit and help needed which are good issues for new contributors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants