[8.0] [MOD-9557] Fix incorrect vector blob size calculation #672
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* add size_t bg_vector_indexing_count;
size_t main_vector_indexing_count;
* fix datasize in executeInsertJob
* disable neighbours print
* replace manual blob size calcaulations
format
comment out new tests
* INT8Test::PopulateRandomVector
increases seed every call so we will get different vectors.
this required a change in metrics_test because now the score is not exactly 0.
VecSimIndexAbstract CastToBruteForce()-> rename to GetFlatBufferIndex
add BruteForceIndex CastToBruteForce()
add getFlatBufferIndexAsBruteForce to tiered index to get BruteForceIndex
functions added to hnsw and BF:
Formalize getDataByLabel definition: populate the intput empty vector with the vectors data,not including any additional meta data that might present in the vector storage like the norm.
add getStoredVectorDataByLabel returns a vector of char vectors. each vector contains the vector as it is stored in the vectors container, including the meta data if exists.
* build test with VERBOSE=1
* verbose cov
* verbose cov fix
assert in FP32SpacesOptimizationTest::FP32InnerProductTest
* fix
* add force_copy to maybeCopyToAlignedMem and preprocessQuery
use preprocessQuery with force_copy in batch iterator instead on preprocessQueryInPlace
* also batch in svs
* revert redundancies
* tiered batch iterator doesn't copy the blob
* add
// TODO: handle original_blob_size != processed_bytes_count
* revert pull request changes
* move getDataByLabel and getStoredVectorDataByLabel to VecSimIndexAbstract
* add test_index_test_utils
currently testing get_stored_vector_data_single_test
* getFlatBufferIndex returns BruteForceIndex
* little test name fix
* small fix
* fix range
(cherry picked from commit 98cf0a1)
meiravgri
requested review from
alonre24 and
GuyAv46
and removed request for
alonre24
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 8.0 #672 +/- ##
==========================================
- Coverage 96.92% 96.71% -0.21%
==========================================
Files 107 107
Lines 5729 5761 +32
==========================================
+ Hits 5553 5572 +19
- Misses 176 189 +13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
GuyAv46
approved these changes
May 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
backport #665 to 8.0
conflict resoultions were related to svs files.