Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try using Lucene IntIntHashMap to speedup and reduce memory usage of top-K counting #662

Open
2 tasks
alexklibisz opened this issue Mar 27, 2024 · 0 comments
Open
2 tasks

Comments

@alexklibisz
Copy link
Owner

Background

Breaking this out of #160, specifically this comment: #160 (comment)

I'd like to try using the IntIntHashMap from Lucene 9.x to decrease latency and memory usage for counting the top-K hits in LSH queries.

Deliverables

  • Implement top-k counting using IntIntHashMap
  • Demonstrate that it's a measurable speedup in benchmarks

Related Issues

So far it looks like the speedup is only noticeable in larger datasets. So this might require expanding benchmarking to datasets larger than Fashion Mnist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant