Make cosine similarity faster by storing magnitude and normalizing vectors #99445
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
benwtrent
added
>enhancement
test-full-bwc
|
Hi @benwtrent, I've created a changelog YAML for you. |
|
Pinging @elastic/es-search (Team:Search) |
|
Did a comparison of this vs. regular Its weird how on search its continuously slightly slower (ran multiple times to confirm). Also the 99.9+% latency on index append is crazy! All these vectors are already normalized. So, I am not 100% sure what could be causing this issue. The comparison should be pretty near 1-to-1 when using this PR with |
benwtrent
added
cloud-deploy
|
@elasticmachine update branch |
jpountz
reviewed
Nov 15, 2023
| if (magnitudeIn == null) { | ||
| return false; | ||
| } | ||
| int currentDoc = magnitudeIn.docID(); |
There was a problem hiding this comment.
if we used advanceExact() we could simplify this logic to just do something like
if (magnitudeIn.advanceExact(docId)) {
magnitude = Float.intBitsToFloat((int) magnitudeIn.longValue());
} else {
magnitude = 1f;
}In bencharks I ran a few years ago, advanceExact() performed noticeably faster than advance because it just needs to check if a doc exists, not compute the next doc.
|
@jpountz my main concern is Here is another run, baseline is |
…sticsearch into feature/make-cosine-faster
|
Thinking about next steps out loud: if we do that then we could deprecate |
jpountz
reviewed
Nov 29, 2023
server/src/main/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapper.java
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapper.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/elasticsearch/index/mapper/vectors/NormalizedCosineFloatVectorValues.java
Outdated
Show resolved
Hide resolved
ChrisHegarty
reviewed
Dec 1, 2023
jpountz
approved these changes
Dec 1, 2023
server/src/main/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapper.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/elasticsearch/index/mapper/vectors/NormalizedCosineFloatVectorValues.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/elasticsearch/index/mapper/vectors/NormalizedCosineFloatVectorValues.java
Outdated
Show resolved
Hide resolved
ChrisHegarty
approved these changes
Dec 1, 2023
server/src/main/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapper.java
Show resolved
Hide resolved
mayya-sharipova
approved these changes
Dec 1, 2023
benwtrent
added
the
auto-merge
mayya-sharipova
added a commit
to mayya-sharipova/elasticsearch
that referenced
this pull request
Jun 22, 2024
PR elastic#99445 introduced automatic normalization of dense vectors with cosine similarity. This adds a note about this in the documentation. Relates to elastic#99445
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jun 22, 2024
mayya-sharipova
added a commit
to mayya-sharipova/elasticsearch
that referenced
this pull request
Jun 22, 2024
PR elastic#99445 introduced automatic normalization of dense vectors with cosine similarity. This adds a note about this in the documentation. Relates to elastic#99445
mayya-sharipova
added a commit
to mayya-sharipova/elasticsearch
that referenced
this pull request
Jun 22, 2024
PR elastic#99445 introduced automatic normalization of dense vectors with cosine similarity. This adds a note about this in the documentation. Relates to elastic#99445
mayya-sharipova
added a commit
to mayya-sharipova/elasticsearch
that referenced
this pull request
Jun 22, 2024
PR elastic#99445 introduced automatic normalization of dense vectors with cosine similarity. This adds a note about this in the documentation. Relates to elastic#99445
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jun 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
cosineis our default similarity and should provide a good experience on speed.dot_productis faster thancosineas it doesn't require calculating vector magnitudes in the similarity comparison loop. Instead, it can assume vectors have a length of1and use an optimizeddot_productcalculation.However,
cosineas it exists today accepts vectors of any magnitude and cannot take advantage of this.This commit addresses this by:
cosine!= 1).dot_productLucene calculationcosinefields