Skip to content

Conversation

@benwtrent
Copy link
Member

This adds bulk scoring to diversity check. While this means that diversity check cannot exit super early (e.g. if it only needs to check 2 docs), I continually see diversity check as being the most expensive part of HNSW graph merging.

This tells me that typically, it isn't just one doc that is checked.

I ran 1M 768 cohere, force-merging with 4 threads.

baseline: 128.01
candidate: 92.15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant