Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GEO] OOM Error when using QuadPrefixTree with 1m precision #9860

Closed
nknize opened this issue Feb 24, 2015 · 0 comments · Fixed by #10652
Closed

[GEO] OOM Error when using QuadPrefixTree with 1m precision #9860

nknize opened this issue Feb 24, 2015 · 0 comments · Fixed by #10652
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes >bug >enhancement

Comments

@nknize
Copy link
Contributor

nknize commented Feb 24, 2015

Fixing issue #9691 revealed a deeper problem with the QuadPrefixTree's memory usage. At 1m precision the example shape in https://gist.github.com/nknize/abbcb87f091b891f85e1 consumes more than 1GB of memory. This is initially alleviated by using 2 bit encoded quads (instead of 1byte) but only delays the problem. Moreover, as new complex shapes are added duplicate quadcells are created - thus introducing unnecessary redundant memory consumption (an inverted index approach makes mosts sense - its Lucene!).

For now, if a QuadTree is used for complex shapes great care must be taken and precision must be sacrificed (something that's automatically done with the distance_error_pct without the user knowing - which is a TERRIBLE approach). An alternative improvement could be to apply a Hilbert R-Tree - which will be explored as a separate issue. Or to restrict the accuracy to a lower level of precision (something that's undergoing experimentation).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes >bug >enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants