v0.17.0

What's Changed

Bug Fixes

• bugfix on fast bitset's AND operator by @LHT129 in #1036
• upgrade robin_map version to v0.14.0 by @inabao in #1040
• fix qps reduce in sindi by @inabao in #1048
• fix incorrect CXX ABI in binary distribution by @wxyucs in #1044
• fix compile error in tests by @inabao in #1058
• fix(sindi): fix the issue where Sindi returns irrelevant results by @inabao in #1085
• fix(hgraph): fix HGraph deserialization failure by @inabao in #1083
• fix(sindi): fix compilation issues of sindi in versions prior to C++17 by @inabao in #1080
• fix(sindi): fix redundant results in the return data by @inabao in #1108
• fix: add missing include in commen.h by @wxyucs in #1127
• fix(deserialize): fix core when deserializing empty index by @inabao in #1150
• fix(index): add check for empty input in index by @inabao in #1159
• fix(int8): fix the missing header files issue by @inabao in #1167
• fix(neon): asan problem on load data by @LHT129 in #1174
• fix(hnsw): performance degradation after remove by @ShawnShawnYou in #1176
• fix(serialize): support empty serialize for export model by @LHT129 in #1169
• fix(hgraph): fix recall issues in HGraph merge by @inabao in #1189
• fix(cmake): fix wrong message on RUNTIME_AVX512VPOPCNTDQ by @LHT129 in #1198
• fix: LabelTable deserialzie error by @LHT129 in #1208
• fix(quantizer): modify the instantiation location of the adapter by @inabao in #1211
• fix(dataset): fix the string deep copy not being properly implemented by @inabao in #1214
• fix(attribute): fix attribute init with use_bucket by @LHT129 in #1216
• fix(hnsw): performance degradation by @ShawnShawnYou in #1222
• fix(attribute): fix loss json init for bruteforce by @LHT129 in #1220

New Features

• add overtime check on search by @LHT129 in #1027
• Introduce transform quantizer by @ShawnShawnYou in #1031
• Support Export All ID from index by @LHT129 in #1069
• support tombstone recovery in hnsw by @ShawnShawnYou in #1056
• Support data export on IVF Index by @LHT129 in #1052
• feat(ivf): support calculate distance by id for IVF by @inabao in #1096
• feat(dataset): support Append and DeepCopy methods in Dataset by @inabao in #1074
• feat(index): support GetDataByIdWithFlag by @LHT129 in #1105
• feat(search): extend search request to cover all search cases by @LHT129 in #1123
• feat(extra info): Support ExtraInfo for all index by @LHT129 in #1135
• feat(HGraph): Support Hgraph's graph storage with different io config by @LHT129 in #1136
• feat(HGraph): add multi-threaded searcher to parallelize single query execution by @mly5269 in #1071
• feat(hgraph): support set topk factor while searching by @inabao in #1180
• feat(sindi): support passing an allocator into KnnSearch by @inabao in #1195

Improvements

• refactor heap merge on ivf parallel search by @LHT129 in #1012
• strip release of pyvsag by @LHT129 in #1021
• add more tag index check on compatibility by @LHT129 in #1024
• add benchmark parameters of popular datasets by @wxyucs in #1005
• add tool to analyze index for hgraph by @inabao in #1025
• increase beam width during reordering by @inabao in #1037
• add file remove check on disk io type by @LHT129 in #1023
• reduce memory on fast_bitset by @LHT129 in #1042
• replace Vector with raw pointer on bitset by @LHT129 in #1067
• support store raw vector on HGraph by @LHT129 in #1063
• support analyze HGraph by searching by @inabao in #1045
• refactor(quantizer): optimize quantizer's compute by @LHT129 in #1089
• feat(reader): add the MultRead method to the reader interface by @inabao in #1082
• refactor: declare shared_ptr/unique_ptr with marco by @LHT129 in #1095
• support in-place update in HGraph by @ShawnShawnYou in #1049
• chore: add more tag for compatibility by @LHT129 in #1101
• refactor(index): reorder the huge index's function by @LHT129 in #1103
• feat(sindi): support several interfaces by @ShawnShawnYou in #1111
• feat(ivf): support for self-checking and query-based checking of IVF inverted indexes by @inabao in #1116
• feat(SINDI): support batch distance calculate by @ShawnShawnYou in #1131
• fix(SINDI): filter performance by @ShawnShawnYou in #1133
• refactor(sindi): remove the dimension check for the sparse vector index by @inabao in #1130
• feat(sindi): support exclude the footer during deserialize by @inabao in #1122
• refactor(headers): improve header include rules by @LHT129 in #1118
• feat(SINDI): support set immutable by @ShawnShawnYou in #1132
• feat(simd): add int8 ip distance impl by @Coien-rr in #1100
• feat(SINDI): support get memory usage by @ShawnShawnYou in #1141
• typo(test): replace "[]" with "()" for catch2 test by @LHT129 in #1143
• feat(sindi): add parameter check for sindi by @inabao in #1144
• typo(index): typo error on index's serialize log by @LHT129 in #1139
• refactor(test): reduce full test size by @LHT129 in #1138
• refactor: use unique_ptr instead shared_ptr by @LHT129 in #1142
• coverage: improve coverage rate by @LHT129 in #1117
• refactor(kmeans): split kmeans compile by @LHT129 in #1137
• refactor: move files by @LHT129 in #1158
• feat(int8): add int8 type support for full-precision quantizer by @Coien-rr in #1152
• refactor(hgraph): simplify hgraph's sparse parameter parsing logic by @inabao in #1164
• feat(HGraph): support tombstone recovery by @ShawnShawnYou in #1160
• refactor(footer): reduce io count when parsing footer by @inabao in #1171
• refactor(option): lower the minimum block size limit by @inabao in #1165
• feat(Hgraph): support tunning alpha in MRNG by @HeHuMing in #1104
• refactor: json optimize compile by @LHT129 in #1168
• refactor(test): clear ci log and cout test name by @inabao in #1182
• tests: speed up functests by reduce dim upperbound by @LHT129 in #1187
• refactor: rename data_cell to datacell by @wxyucs in #1178
• chore(cmake): clear CMake warnings, add license to the file header by @wxyucs in #1193
• refactor(attr): use parameter to create attribute datacell by @LHT129 in #1200
• refactor: split unittest compile to speedup link by @LHT129 in #1202
• feat(quantization): Introduce QuantizerAdapter to support non-FP32 data types in quantizers by @Coien-rr in #1185
• tests: add serialize check on export model test by @LHT129 in #1203
• feat: support use old serial format by @wxyucs in #1201
• tests: relax recall threshold to fit random dataset by @LHT129 in #1204

Other Changes

• update ivf doc by @LHT129 in #1011
• fix label_remap size serialize by @shadowao in #1022
• [skip ci] update document of hgraph by @inabao in #1032
• update perf.yml for COHERE by @inabao in #1038
• support sve to accelerate distance compute by @jac0626 in #989
• support mrng in hgraph by @HeHuMing in #1079
• [skip ci] docs(hgraph): refactor HGraph doc by @LHT129 in #1102
• test(ivf): refactor ivf fp16 test on cosine metric by @LHT129 in #1106
• docs(ivf): add parameter table for ivf doc by @LHT129 in #1120
• [skip ci] docs(brute_force): add brute_force doc by @LHT129 in #1121
• complete missing neon fuctions by @jac0626 in #1018
• feat(sparse): improve performance of batch distances calculation by @ShawnShawnYou in #1192
• [skip ci] doc: modify the formatting of publications and references by @inabao in #1207
• refactor(reader_io): support reader_io omits I/O operations during deserialization by @inabao in #1212

New Contributors

• @HeHuMing made their first contribution in #1079

Full Changelog: v0.16.0...v0.17.0