The index size is wrong #2890
Assignees
Labels
kind/bug
Issues or changes related a bug
Comments
Merged
mergify bot
pushed a commit
that referenced
this issue
Jul 21, 2020
* fix update index size Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add GetSize() interface 4 SPTAG Signed-off-by: cmli <chengming.li@zilliz.com> * fix index size of index hnsw, annoy and nsg Signed-off-by: cmli <chengming.li@zilliz.com> * fix clang Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add sdk 4 non-ivf index test Signed-off-by: cmli <chengming.li@zilliz.com> * remove std::cout and make clang-format Signed-off-by: cmli <chengming.li@zilliz.com> * fix cout Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * checkout sdk Signed-off-by: cmli <chengming.li@zilliz.com> * fix binary ivf Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix hnsw Signed-off-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
mergify bot
pushed a commit
that referenced
this issue
Jul 21, 2020
* fix NM index size Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add raw data size of hnsw_nm and nsg_nm Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com>
fishpenguin
added a commit
that referenced
this issue
Jul 25, 2020
* fix a typo (#2712) Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> * snapshot integration (#2716) * code opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add some APIs for SSDBImpl Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * Web fix 2690 (#2691) * Remove body parser in show partitions endpoints (fix #2690) Signed-off-by: yhz <413554850@qq.com> * Remove surplus output and comments Signed-off-by: yhz <413554850@qq.com> * Remove surplus output and comments Signed-off-by: yhz <413554850@qq.com> * Remove space line Signed-off-by: yhz <413554850@qq.com> * Remove surplus ut Signed-off-by: yhz <413554850@qq.com> * laten sleep time in case test_delete_vector_collection_count_no_flush Signed-off-by: yhz <413554850@qq.com> * #2689 Construct Knowhere Index Without Data (#2704) * Offset for IVFFlat Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * modify segment reader Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * add index_flat_nm Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * fix compilation issue Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix segment reader Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * Fix issue Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix hnsw_nm Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * fix nsg Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * fix bug Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * NSG Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Fix NSG issue Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * client test Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix index size Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * remove unnecessary things Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add changelog Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Fix clang format Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add changelog Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Fix compile error Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Fix compile error Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Fix compile error Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Fix compile error Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Fix issues Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * Change data to shared_ptr Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix hnsw and nsg ut Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * clang-format Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * not to insert cache immediately Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * Fix macro issue Signed-off-by: sahuang <xiaohai.xu@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com> * (db/snapshot): Code Integration (#2718) * (db/snapshot): integrate stage 1 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): integrate stage 2 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): integrate stage 3 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix ut Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): integrate stage 4 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): integrate stage 5 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): integrate stage 6 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): integrate stage 7 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add params for some resources Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add field type Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): implement iterate framework for Snapshot Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): refactor code Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint errors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add SnapshotHandlers Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add SnapshotVistor Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add SizeField for all resource commit Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add SegmentsToSearch collector Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Iterate handler update Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Update IterateHandler Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add some hooks for IterateHandler Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Small code refactor Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add const for some snapshot APIs Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add const for some snapshots APIs Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Update create collection Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): merge to new ssdb Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint errors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add test_db in ssdb Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update test_ssdb Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add PartitionTest Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add DropIndexOperation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add IndexTest Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Fix some bugs Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add RowCount for some resources Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): code refactor Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix compile error Signed-off-by: peng.xu <peng.xu@zilliz.com> * Add pipeline step to print/get the container logs (#2730) * Add pipeline step to print/get the container logs Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Add pipeline step to print/get the container logs Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Create index failed caused by server crashed (#2736) Signed-off-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: groot <yhmo@zeronedata.com> * (db/snapshot): Add DropAllIndexOperation and related UT (#2737) * (db/snapshot): add drop all index operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add ut for drop all index operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * search combine configable Signed-off-by: groot <yihua.mo@zilliz.com> * typo Signed-off-by: groot <yihua.mo@zilliz.com> * Increase GRPC concurrency (#2671) * Increase GRPC concurrency Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * Fix lint Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * Add limitation Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * Fix compilation error Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * Fix compilation error Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * Fix compilation error Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * Change Milvus Helm Version to v0.10.1 (#2678) Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * [skip ci] update nightly cron (#2680) * Change Milvus Helm Version to v0.10.1 Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * [skip ci] update nightly cron Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Update nightly cron (#2681) * Change Milvus Helm Version to v0.10.1 Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * [skip ci] update nightly cron Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * [skip ci] update nightly cron Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * * #2642 Create index failed caused by server crashed. Signed-off-by: yhmo <yihua.mo@zilliz.com> * changelog Signed-off-by: yhmo <yihua.mo@zilliz.com> * changelog Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * enable ivf_pq <ip> search cases on GPU (#2708) * update cron on 0.10.1 Signed-off-by: zw <zw@milvus.io> * enable ivf_pq <ip> search cases on GPU Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> * * #2642 Create index failed caused by server crashed. Signed-off-by: yhmo <yihua.mo@zilliz.com> * changelog Signed-off-by: yhmo <yihua.mo@zilliz.com> * changelog Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * #2692 Milvus hangs during multi-thread concurrent search Signed-off-by: yhmo <yihua.mo@zilliz.com> * Fix compiling error Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * [skip ci] Update PR template Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * Let codec singleton (#2742) Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * (db/snapshot): Bug fix in operations and unittest (#2744) * (db/snapshot): fix bugs and update UT Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update Snapshot ToString Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add DropIndex API Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add DropIndex API UT Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add more APIs for Snapshot (#2747) * (db/snapshot): Add more APIs for Snapshot Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Refactor field element related code Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove some prints Signed-off-by: peng.xu <peng.xu@zilliz.com> * add background threads to SSDBImpl (#2751) * add background threads to SSDBImpl Signed-off-by: yhmo <yihua.mo@zilliz.com> * code format Signed-off-by: yhmo <yihua.mo@zilliz.com> * snapshot integrate GetEntityByID (#2753) * code opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add some APIs for SSDBImpl Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * partially add GetVectorById Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * snapshot opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix typo Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update GetVectorByID framework Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * rename GetResFiles to GetResPath Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update GetVectorByIdSegmentHandler Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add GetEntityByID Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update DataType Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update ParamField Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix: upgrade com.alibaba:fastjson from 1.2.70 to 1.2.71 (#2756) Snyk has created this PR to upgrade com.alibaba:fastjson from 1.2.70 to 1.2.71. See this package in NPM: https://www.npmjs.com/package/com.alibaba:fastjson See this project in Snyk: https://app.snyk.io/org/cxie/project/2f701ce6-b421-4bbf-aaad-9d2bdb8569e4?utm_source=github&utm_medium=upgrade-pr * (db/snapshot): Implement operation rollback (#2760) * (db/snapshot): refactor Operations Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): refactor Operations Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add rollback for operations Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot) snapshot integrate (#2770) * code opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add some APIs for SSDBImpl Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * partially add GetVectorById Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * snapshot opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix typo Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update GetVectorByID framework Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * rename GetResFiles to GetResPath Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update GetVectorByIdSegmentHandler Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add GetEntityByID Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update DataType Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update ParamField Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update GetEntityByIDSegmentHandler Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * support INT8 and INT16 in GetEntityById Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add GetIDsInSegment Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update FieldType Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add InsertEntities and DeleteEntities Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add HybridQuery Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * (db/snapshot): Add some helper visitors (#2777) * (db/snapshot): add Segment related visitors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * Update template Signed-off-by: jinhai <hai.jin@zilliz.com> * snapshot integrate (#2782) * update HybridQuery Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add IterateHandler Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * IterateHandler opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * rewrite mem manager 1 (#2762) * rewrite mem manager Signed-off-by: yhmo <yihua.mo@zilliz.com> * rewrite mem manager 2 Signed-off-by: yhmo <yihua.mo@zilliz.com> * refine Signed-off-by: yhmo <yihua.mo@zilliz.com> * rewrite mem manager 3 Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * merge manager 1 Signed-off-by: yhmo <yihua.mo@zilliz.com> * fix too many data copies (#2784) Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * fix a bug of getting wrong nprobe limitation in knowhere on GPU version (#2788) * fix a bug of getting wrong nprobe limitation in knowhere on GPU version Signed-off-by: cmli <chengming.li@zilliz.com> * update changelog and fix build error Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> * Hnsw sq8 support (#2799) * modify for SQ8 Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * add hnsw-sq8-support Signed-off-by: cmli <chengming.li@zilliz.com> * make clang-format and update changelog Signed-off-by: cmli <chengming.li@zilliz.com> * fix lint error Signed-off-by: cmli <chengming.li@zilliz.com> * hide unittest error temporary Signed-off-by: cmli <chengming.li@zilliz.com> * fix ut error Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> * rewrite merge manager 1 (#2797) * rewrite merge manager 1 Signed-off-by: yhmo <yihua.mo@zilliz.com> * rewrite merge manager 2 Signed-off-by: yhmo <yihua.mo@zilliz.com> * (db/snapshot): Add some visitors (#2800) * (db/snapshot): add more visitors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add some Iterators Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update visitors Signed-off-by: peng.xu <peng.xu@zilliz.com> * fix #2752 (#2789) * fix #2752 Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> (cherry picked from commit d7c47a7) * #2802 Add new index: IVFSQ8NR (#2805) * support new index ivfsq8nr Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * remove timeout Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix task_search (#2808) * fix task_search Signed-off-by: cqy <yaya645@126.com> * fix unit test Signed-off-by: cqy <yaya645@126.com> * add unittest 4 hnsw_sq8nr and fix some bug in passing (#2819) * modify for SQ8 Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * add hnsw-sq8-support Signed-off-by: cmli <chengming.li@zilliz.com> * make clang-format and update changelog Signed-off-by: cmli <chengming.li@zilliz.com> * fix lint error Signed-off-by: cmli <chengming.li@zilliz.com> * hide unittest error temporary Signed-off-by: cmli <chengming.li@zilliz.com> * fix ut error Signed-off-by: cmli <chengming.li@zilliz.com> * add unittest 4 hnsw_sq8nr and fix several bugs Signed-off-by: cmli <chengming.li@zilliz.com> * fix lint error Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> * snapshot integrate (#2814) * update HybridQuery Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add IterateHandler Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * IterateHandler opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SSSegmentReader Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add src/codecs/snapshot Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update latest code Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update SSSegmentReader interface Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SSSegmentWriter Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update SSSegmentWriter interface Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update unittest Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix clang-format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix unittest/db build fail Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * use SSSegmentReader and SSSegmentWriter in SSDBImpl Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix clang-format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix build issue Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * Add SDK support for IVFSQ8NR (#2821) * add c++ sdk Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix clang Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * (db/snapshot): LSN logic change and design EventHandlerFactory (#2815) * (db/snapshot): add more visitors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add some Iterators Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update visitors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add event handler factory Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): process lsn Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): small change Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): precheck duplicate collection in create collection operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * [skip ci] Update README Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * [skip ci] Update README_CN Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * snapshot integrate (#2830) * update HybridQuery Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add IterateHandler Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * IterateHandler opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * opt Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SSSegmentReader Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add src/codecs/snapshot Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update latest code Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update SSSegmentReader interface Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SSSegmentWriter Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update SSSegmentWriter interface Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update unittest Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix clang-format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix unittest/db build fail Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * use SSSegmentReader and SSSegmentWriter in SSDBImpl Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix clang-format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix build issue Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add test_segment Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * (db/snapshot): update visitors (#2837) * (db/snapshot): add more visitors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add some Iterators Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update visitors Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add event handler factory Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): process lsn Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): small change Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): precheck duplicate collection in create collection operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update visitor Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update visitor 2 Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update visitor 3 Signed-off-by: peng.xu <peng.xu@zilliz.com> * Fix rnsg master (#2831) * fix rnsg ip Signed-off-by: cqy <yaya645@126.com> * fix rnsg search in master Signed-off-by: cqy <yaya645@126.com> * Hnsw sq8 support (#2835) * modify for SQ8 Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * add hnsw-sq8-support Signed-off-by: cmli <chengming.li@zilliz.com> * add unittest 4 hnsw_sq8nr and fix several bugs Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> * remove old api (#2833) * remove old api Signed-off-by: yhmo <yihua.mo@zilliz.com> * refine insert code Signed-off-by: yhmo <yihua.mo@zilliz.com> * refine code Signed-off-by: yhmo <yihua.mo@zilliz.com> * Update sqlite_6_to_4.sql (#2832) Simplify SQL statements Signed-off-by: dddddai <853238012@qq.com> * (db/snapshot) Add snapshot DB module (#2851) * New db impl Signed-off-by: yhz <413554850@qq.com> * Add db impl for snapshot Signed-off-by: yhz <413554850@qq.com> * Update operations step holders Signed-off-by: yhz <413554850@qq.com> * Update snapshot dbimpl & create collection test pass Signed-off-by: yhz <413554850@qq.com> * Delete operation pass Signed-off-by: yhz <413554850@qq.com> * Add constraints for db table Signed-off-by: yhz <413554850@qq.com> * Update index operation steps behavior Signed-off-by: Yhz <yinghao.zou@zilliz.com> * Add lock in mysql engine and remove lock in store Signed-off-by: yhz <413554850@qq.com> * Index unittest case pass Signed-off-by: yhz <413554850@qq.com> * Add merge operation step context Signed-off-by: yhz <413554850@qq.com> * Remove constraints in DB and snapshot ut pass Signed-off-by: yhz <413554850@qq.com> * update DBImple Signed-off-by: Yhz <yinghao.zou@zilliz.com> * Add DBHelper to generate sql & add mock Mem db & snaoshot ut pass Signed-off-by: Yhz <yinghao.zou@zilliz.com> * Add mock db engine Signed-off-by: Yhz <yinghao.zou@zilliz.com> * Update DB select API using template for resource Signed-off-by: Yhz <yinghao.zou@zilliz.com> * Update snapshot db name and move them under folder meta Signed-off-by: yhz <413554850@qq.com> * clang-format Signed-off-by: yhz <413554850@qq.com> * redefine SegmentWriter (#2853) * redefine SegmentWriter Signed-off-by: yhmo <yihua.mo@zilliz.com> * redefine SegmentWriter Signed-off-by: yhmo <yihua.mo@zilliz.com> * refine code Signed-off-by: yhmo <yihua.mo@zilliz.com> * move index type try2 (#2860) * move EngineType and MetricType to MetaTypes.h Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * move IndexType Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update changelog Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update changelog Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * snapshot integrate (#2854) * add test_segment Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update interface GetResPath Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * retry ci Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update SSSegmentWriter Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * #2802 Add new index: IVFSQ8NR_NM (#2865) * add ivfsq8nr_nm Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * refactor Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix error Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * [skip ci] remove unused param Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * (db/snapshot): Update row count related logic (#2862) * (db/snapshot): update for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix bug in NewSegmentOperation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove dummy print Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add some test for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * add MILVUS_SUPPORT_SPTAG for some code (#2866) * add MILVUS_SUPPORT_SPTAG for some code Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * use knowhere::IndexEnum in db Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * keep EngineType order Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * # 2828 opt faiss build (#2855) * add FAISS_USE_FLOAT16 Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix faiss build Signed-off-by: cqy <yaya645@126.com> Co-authored-by: yudong.cai <yudong.cai@zilliz.com> * (db/snapshot): update size related logic (#2877) * (db/snapshot): update for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix bug in NewSegmentOperation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove dummy print Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add some test for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic related ut Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): rollback if operation is not done Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): clean store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove some dependency Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot) Add snapshot DB unittest (#2856) * (snapshot) Add snapshot DB unittest Signed-off-by: yhz <413554850@qq.com> * Optimize mock db Signed-off-by: yhz <413554850@qq.com> * correct server context file name Signed-off-by: yhz <413554850@qq.com> * Fix Prometheus rebuild problem (#2878) * Solve prometheus repeatedly build problem Signed-off-by: yangxuan <xuan.yang@zilliz.com> * recover build.sh Signed-off-by: yangxuan <xuan.yang@zilliz.com> * refactor build.sh Signed-off-by: yangxuan <xuan.yang@zilliz.com> * altering changelog Signed-off-by: yangxuan <xuan.yang@zilliz.com> * rm not-useful comments Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> * Add some cpp unittests about scheduler algorithm (#2888) * add some testcases Signed-off-by: yangxuan <xuan.yang@zilliz.com> * reformat code Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> (cherry picked from commit 8135cee) * opt faiss build (#2883) * opt faiss build Signed-off-by: cqy <yaya645@126.com> * use BlockingQueue in JobMgr (#2885) * using BlockingQueue in JobMgr Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix unittest Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * segment writer (#2873) * redefine SegmentWriter Signed-off-by: yhmo <yihua.mo@zilliz.com> * redefine SegmentWriter Signed-off-by: yhmo <yihua.mo@zilliz.com> * refine code Signed-off-by: yhmo <yihua.mo@zilliz.com> * mem insert Signed-off-by: yhmo <yihua.mo@zilliz.com> * unittest Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * fix Signed-off-by: yhmo <yihua.mo@zilliz.com> * fix some issues Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * segment reader Signed-off-by: yhmo <yihua.mo@zilliz.com> * segment writer Signed-off-by: yhmo <yihua.mo@zilliz.com> * remove files Signed-off-by: yhmo <yihua.mo@zilliz.com> * remove code Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * merge Signed-off-by: yhmo <yihua.mo@zilliz.com> * fix merge Signed-off-by: yhmo <yihua.mo@zilliz.com> * get entities Signed-off-by: yhmo <yihua.mo@zilliz.com> * Fixed broken links. (#2902) Signed-off-by: PahudPlus <haimeng.cai@zilliz.com> * Set MySQL host ip to 127.0.0.1 in jenkins unittest step (#2905) Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Fix warning master (#2894) * fix clean unused variable warning in knowhere Signed-off-by: cqy <yaya645@126.com> * clean unused variable warning in knowhere Signed-off-by: cqy <yaya645@126.com> * clean unused variable warning in knowhere Signed-off-by: cqy <yaya645@126.com> * clean unused variable warning in knowhere Signed-off-by: cqy <yaya645@126.com> * fix konwhere warning Signed-off-by: cqy <yaya645@126.com> * fix #2768 (#2809) * fix #2752 Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * fix #2768 Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: groot <yihua.mo@zilliz.com> * typo again Signed-off-by: groot <yihua.mo@zilliz.com> * [0.10.1] update shards ci && enable pq case (#2804) * update cron on 0.10.1 Signed-off-by: zw <zw@milvus.io> * enable ivf_pq <ip> search cases on GPU Signed-off-by: zw <zw@milvus.io> * [skip ci] debug Signed-off-by: zw <zw@milvus.io> * [skip ci] add artifacts Signed-off-by: zw <zw@milvus.io> * update shards ci && enable pq case Signed-off-by: zw <zw@milvus.io> * update assertion for pq Signed-off-by: zw <zw@milvus.io> * disable rnsg: issue #2813 Signed-off-by: zw <zw@milvus.io> * add binary version in log file name Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> * Add python functional tests about timeout (#2793) * Add python tests about param timeout Signed-off-by: yangxuan <jumpthepig@gmail.com> * add faiss configure args msg print Signed-off-by: yangxuan <xuan.yang@zilliz.com> * change assertion conditions Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> * add testcase of empty vector insert (#2850) * add testcase of empty vector insert Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * format python code Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * (db/snapshot): Update Store (#2897) * (db/snapshot): update for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix bug in NewSegmentOperation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove dummy print Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add some test for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic related ut Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): rollback if operation is not done Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): clean store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove some dependency Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update for store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update Store.h Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update store related code Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add field element modification operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): change new operation name Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add AddSegmentFileOperation (#2924) * (db/snapshot): update for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix bug in NewSegmentOperation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove dummy print Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add some test for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic related ut Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): rollback if operation is not done Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): clean store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove some dependency Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update for store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update Store.h Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update store related code Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add field element modification operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): change new operation name Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add Segment File Operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * fix: upgrade com.alibaba:fastjson from 1.2.71 to 1.2.72 (#2925) Snyk has created this PR to upgrade com.alibaba:fastjson from 1.2.71 to 1.2.72. See this package in NPM: https://www.npmjs.com/package/com.alibaba:fastjson See this project in Snyk: https://app.snyk.io/org/cxie/project/2f701ce6-b421-4bbf-aaad-9d2bdb8569e4?utm_source=github&utm_medium=upgrade-pr * (db/snapshot) Optimize snapshot mock db (#2896) * Add mysql time statices Signed-off-by: yhz <413554850@qq.com> * Add detail timestamp recorder Signed-off-by: yhz <413554850@qq.com> * Optimize snapshot mock DB Signed-off-by: yhz <413554850@qq.com> * Remove surplus space Signed-off-by: yhz <413554850@qq.com> * Updat meta test Signed-off-by: yhz <413554850@qq.com> * Rearrange core cmakelists (#2900) * sort out cmake commands in before project Signed-off-by: yangxuan <xuan.yang@zilliz.com> * update Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com> * rearrange core CmakeLists.txt Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix conflict Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix bugs on ENABLE_FIU Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: Xiangyu Wang <xy.wang@zilliz.com> * add web_impl testcase (#2920) Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * [skip ci] Update cases for the resources changed (#2907) * update cron on 0.10.1 Signed-off-by: zw <zw@milvus.io> * enable ivf_pq <ip> search cases on GPU Signed-off-by: zw <zw@milvus.io> * [skip ci] debug Signed-off-by: zw <zw@milvus.io> * [skip ci] add artifacts Signed-off-by: zw <zw@milvus.io> * update shards ci && enable pq case Signed-off-by: zw <zw@milvus.io> * update assertion for pq Signed-off-by: zw <zw@milvus.io> * disable rnsg: issue #2813 Signed-off-by: zw <zw@milvus.io> * add binary version in log file name Signed-off-by: zw <zw@milvus.io> * enable rnsg case Signed-off-by: zw <zw@milvus.io> * [skip ci] Update config cases for the resources changed Signed-off-by: zw <zw@milvus.io> * [skip ci] Fix limit Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> * refine code (#2927) * refine code Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * excution engine Signed-off-by: yhmo <yihua.mo@zilliz.com> * (db/snapshot): add GC event and implement crtp for base resource (#2926) * (db/snapshot): update for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix bug in NewSegmentOperation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove dummy print Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add some test for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic related ut Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): rollback if operation is not done Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): clean store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove some dependency Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update for store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update Store.h Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update store related code Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add field element modification operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): change new operation name Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add Segment File Operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): crtp for BaseResource Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add InActiveResourcesGCEvent Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix ut error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): small change Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot) Add root path for snapshot GC (#2936) * Add root path for snapshot GC Signed-off-by: yhz <413554850@qq.com> * Add subfolder db in root path Signed-off-by: yhz <413554850@qq.com> * #2890 Fix wrong index size (#2930) * fix update index size Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add GetSize() interface 4 SPTAG Signed-off-by: cmli <chengming.li@zilliz.com> * fix index size of index hnsw, annoy and nsg Signed-off-by: cmli <chengming.li@zilliz.com> * fix clang Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add sdk 4 non-ivf index test Signed-off-by: cmli <chengming.li@zilliz.com> * remove std::cout and make clang-format Signed-off-by: cmli <chengming.li@zilliz.com> * fix cout Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * checkout sdk Signed-off-by: cmli <chengming.li@zilliz.com> * fix binary ivf Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * fix hnsw Signed-off-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com> * scheduler support snapshot (#2944) * using BlockingQueue in JobMgr Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update changelog Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update SSDBImpl::HybridQuery Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add QueryTest Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SearchJob interface Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SSSearchJob and SSSearchTask Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update AddSegmentVisitor Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SSBuildIndexTask and SSBuildIndexJob Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add SSTestTask Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add test_ss_job and test_ss_task Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix clang-format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update TaskCreator Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * #2890 Fix wrong index size (#2942) * fix NM index size Signed-off-by: sahuang <xiaohai.xu@zilliz.com> * add raw data size of hnsw_nm and nsg_nm Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> * Minor editorial updates. (#2946) Signed-off-by: Amy Hong <yun.hong@zilliz.com> * fix cuda version requirement (#2909) Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com> * snapshot scheduler (#2961) * update interface Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add test_ss_job Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update Query Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add dir_root for SSSearchJob and SSBuildIndexJob Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * (db/snapshot): Update DropAllIndexOperation (#2947) * (db/snapshot): update for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix bug in NewSegmentOperation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove dummy print Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add some test for row count Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update size logic related ut Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): rollback if operation is not done Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): clean store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): remove some dependency Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update for store Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update Store.h Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update store related code Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add field element modification operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): change new operation name Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): Add Segment File Operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): crtp for BaseResource Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): add InActiveResourcesGCEvent Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix ut error Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): small change Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update snapshot segmentcommit operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update drop all index operation Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): update ut Signed-off-by: peng.xu <peng.xu@zilliz.com> * (db/snapshot): fix lint error Signed-off-by: peng.xu <peng.xu@zilliz.com> * fix a bug of annoy search parameter exisitence check (#2958) Signed-off-by: cmli <chengming.li@zilliz.com> Co-authored-by: cmli <chengming.li@zilliz.com> * (db/snapshot) Add root path for snapshot store (#2945) * Add root path for snapshot GC Signed-off-by: yhz <413554850@qq.com> * Change Fun name from GetResPath to GetRootPath Signed-off-by: yhz <413554850@qq.com> * Fix the result merging of IVF_PQ IP (#2955) Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * Fix C++ sdk DataType Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix collection test cases bugs Signed-off-by: fishpenguin <kun.yu@zilliz.com> * (db/snapshot) Inactivate gc event (#2960) * Add API to Get inactivate res in Store Signed-off-by: yhz <413554850@qq.com> * Update select logic in MetaAdapter Signed-off-by: yhz <413554850@qq.com> * Add event test case Signed-off-by: yhz <413554850@qq.com> * Add deactive collection test in SSEventTest.TestInActiveResGcEvent Signed-off-by: yhz <413554850@qq.com> * Add CollectinCommit in event test Signed-off-by: yhz <413554850@qq.com> * Add more res in event test Signed-off-by: yhz <413554850@qq.com> * Set url to mock in ssdb test Signed-off-by: yhz <413554850@qq.com> * snapshot build index (#2939) * refine code Signed-off-by: yhmo <yihua.mo@zilliz.com> * refine Signed-off-by: yhmo <yihua.mo@zilliz.com> * ui error Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * build index Signed-off-by: yhmo <yihua.mo@zilliz.com> * add utils Signed-off-by: yhmo <yihua.mo@zilliz.com> * build index Signed-off-by: yhmo <yihua.mo@zilliz.com> * build index Signed-off-by: yhmo <yihua.mo@zilliz.com> * execution engine Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * typo Signed-off-by: yhmo <yihua.mo@zilliz.com> * build error Signed-off-by: yhmo <yihua.mo@zilliz.com> * Add options to build.sh for cuda arch Issue#2940 (#2950) * add option -s in build.sh for cuda arch Signed-off-by: yangxuan <xuan.yang@zilliz.com> * modify changelog Signed-off-by: yangxuan <xuan.yang@zilliz.com> * modify ci configs about cuda arch Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix bug Signed-off-by: yangxuan <xuan.yang@zilliz.com> * modify ci configs about cuda arch in jenkins ci Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * fix bug Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * fix bug Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * modify changelog Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix bug Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * fix bug Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * fix bugs Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix cmake cannot recognize space bug Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix conflict Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com> * Fix test_rpc bugs Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix test_rpc Signed-off-by: fishpenguin <kun.yu@zilliz.com> Co-authored-by: op-hunter <ophunter52@gmail.com> Co-authored-by: cmli <chengming.li@zilliz.com> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com> Co-authored-by: Xiaohai Xu <xiaohai.xu@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: XuPeng-SH <xupeng3112@163.com> Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com> Co-authored-by: groot <yhmo@zeronedata.com> Co-authored-by: groot <yihua.mo@zilliz.com> Co-authored-by: Jin Hai <hai.jin@zilliz.com> Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com> Co-authored-by: zw <zw@milvus.io> Co-authored-by: Snyk bot <snyk-bot@snyk.io> Co-authored-by: cqy123456 <39671710+cqy123456@users.noreply.github.com> Co-authored-by: dwq <41563853+dddddai@users.noreply.github.com> Co-authored-by: XuanYang-cn <51370125+XuanYang-cn@users.noreply.github.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: PahudPlus <64403786+PahudPlus@users.noreply.github.com> Co-authored-by: godchen0212 <67679556+godchen0212@users.noreply.github.com> Co-authored-by: Xiangyu Wang <xy.wang@zilliz.com> Co-authored-by: AmyYH <68527082+AmyYH@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Please state you issue using the following template and, most importantly, in English.
Describe the bug
Now, the system is using index file size to indicate its size in memory.
But sometimes it is a terrible mistake. Please refer to issue #2803
Thus, it will cause the cache policy not to work well.
Steps/Code to reproduce behavior
none
Expected behavior
Correct the index size.
Environment details
Screenshots
none
Additional context
Here is the wrong logic.
DefaultVectorIndexFormat::read_internal()
index->SetIndexSize(length);
The text was updated successfully, but these errors were encountered: