Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(vector): Add heap to neighbour edges #9122

Merged
merged 7 commits into from
Aug 13, 2024
Merged

Conversation

harshil-goel
Copy link
Contributor

@harshil-goel harshil-goel commented Jul 31, 2024

We now store the list of neighbours in a sorted distance from the starting node. This improves the accuracy and the time of the vector.
We are also using findStaticValue() function to get data faster.

Benchmarks:

  1. Dataset 1 : 10k vector 100 dimension
    a. Time to ingest : 6 seconds
    b. Query time: 900us
    c. Accuracy: 97.9%

  2. Dataset 2: 1million vectors 1280 dimension
    a. Time to ingest: 7 mins
    b. Query time (without cache): 35 ms
    c. Query time (with cache): 5 ms

@harshil-goel harshil-goel requested a review from a team as a code owner July 31, 2024 15:28
@CLAassistant
Copy link

CLAassistant commented Jul 31, 2024

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ harshil-goel
❌ Harshil goel


Harshil goel seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

1 similar comment
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ harshil-goel
❌ Harshil goel


Harshil goel seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@dgraph-bot dgraph-bot added area/core internal mechanisms go Pull requests that update Go code labels Jul 31, 2024
@harshil-goel harshil-goel changed the base branch from main to harshil-goel/single_key August 13, 2024 00:05
@harshil-goel harshil-goel changed the base branch from harshil-goel/single_key to main August 13, 2024 00:07
@harshil-goel harshil-goel changed the title perf(vector): Improve how vector index interacts with badger data perf(vector): Improve vector performance Aug 13, 2024
jairad26
jairad26 previously approved these changes Aug 13, 2024
tok/hnsw/helper.go Outdated Show resolved Hide resolved
tok/hnsw/helper.go Show resolved Hide resolved
jairad26
jairad26 previously approved these changes Aug 13, 2024
@harshil-goel harshil-goel changed the title perf(vector): Improve vector performance perf(vector): Add heap to neighbour edges Aug 13, 2024
@harshil-goel harshil-goel merged commit d6ac87e into main Aug 13, 2024
11 of 13 checks passed
@harshil-goel harshil-goel deleted the harshil-goel/vector-fast branch August 13, 2024 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core internal mechanisms go Pull requests that update Go code
Development

Successfully merging this pull request may close these issues.

4 participants