Skip to content

v1.1.0

Latest

Choose a tag to compare

@wxyucs wxyucs released this 24 Oct 10:29
· 44 commits to main since this release
e18db3e

What's Changed

Bug Fixes

  • fix(datacell, odescent): fix multiple out-of-bounds array accesses by @LHT129 in #2364
  • fix: preserve pyramid reorder search stats by @jac0626 in #2416
  • fix(hgraph): drain parallel build futures on worker errors by @jac0626 in #2353
  • fix(sindi): use sparse immutable serialization by @Roxanne0321 in #2437
  • fix(hgraph): tune from available fp32 codes by @jac0626 in #2466
  • fix: restore filter->CheckValid() in visit condition to fix recall drop by @LHT129 in #2462
  • fix(ci): prevent PR lint from being silently skipped by @wxyucs in #2483
  • fix: resolve SINDI clang-tidy failures by @wxyucs in #2477
  • fix(ci): reduce nested OpenBLAS build parallelism by @wxyucs in #2504
  • fix(hgraph): remove ef_search topk upper bound by @wxyucs in #2515
  • fix: avoid virtual resize during PointsMutex construction by @wxyucs in #2524
  • fix(ci): build performance evaluation tool by @vsag-bot in #2535
  • fix: Check Compatibility workflow failed without job logs by @vsag-bot in #2537
  • fix(eval): correct knn latency and QPS measurements by @jac0626 in #2516
  • fix: HGraph disabled-reasoning overhead ratio exceeds test threshold by @vsag-bot in #2536
  • fix: avoid unaligned hnsw link access by @jac0626 in #1989
  • fix(simd): prevent 16-bit accumulator overflow in PQ-FastScan kernels by @jfeng18 in #2284
  • fix(pyramid): remove immutable state shadowing in SetImmutable by @LHT129 in #2549
  • fix: make mmap remap exception-safe on macOS by @LHT129 in #2400
  • fix(hgraph): reset state after removing all elements by @LHT129 in #2580
  • fix: IVF nearest-partition basic test returns an unexpected class ID by @vsag-bot in #2595
  • fix: use single-threaded build for OpenBLAS to avoid race condition by @LHT129 in #2600
  • fix: IVF nearest-partition serialization test returns a different class ID by @vsag-bot in #2564
  • fix: Lint fails on narrowing long-to-int conversions in SINDI by @vsag-bot in #2608
  • fix: TSAN build fails in bundled OpenBLAS build by @vsag-bot in #2618
  • fix(pyramid): stop range trimming at an empty heap by @jac0626 in #2623
  • fix: reorder IVF custom-distance filters by @LHT129 in #2613
  • fix(build): upgrade bundled OpenBLAS to 0.3.24 by @jac0626 in #2459
  • test: stabilize filtered range search recall validation by @jac0626 in #2644
  • fix(ivf): drain add tasks after allocation failure by @LightWant in #2653
  • fix(hgraph): set batch distance result dimensions by @LHT129 in #2648
  • fix(ivf): expose routing HGraph parameters by @LHT129 in #2666
  • fix(ivf): routing HGraph thread count and thread pool issues by @LHT129 in #2665
  • fix(sindi): skip zero-distance range candidates by @Roxanne0321 in #2678
  • fix: include cstdint in binaryset header by @wxyucs in #2699
  • fix(quantization): validate SQ4 uniform truncation rate by @CharlesXu-HQ in #2696
  • fix(hgraph): enable MRLE split RaBitQ fast build by @LightWant in #2739
  • fix: IVF GNO-IMI build fails recall threshold by @vsag-bot in #2744
  • fix(ci): harden changed-source lint filtering by @wxyucs in #2788
  • fix(ci): trust exact workspace for lint Git calls by @wxyucs in #2797
  • fix(build): scope VSAG compile flags by @wxyucs in #2807
  • fix(ci): retain Coverage failure diagnostics by @wxyucs in #2836
  • fix(pyramid): avoid exclusive lock for child lookup by @jac0626 in #2875
  • fix(hgraph): prevent stale entry point after force-remove tail nodes by @LHT129 in #2902
  • fix(test): preserve async I/O mapping profiles by @wxyucs in #2877
  • fix: HGraph supplement I/O mapping and accepted-task draining tests fail by @vsag-bot in #2918
  • fix(python): align SINDI tests with partial statistics by @wxyucs in #2929
  • fix(test): stabilize IVF centroid scan ordering fixture by @wxyucs in #2930
  • fix: correct ASan build reporting and compile attribution by @wxyucs in #2954
  • fix: unify native distance-by-id behavior across indexes by @LHT129 in #2924
  • fix(test): tolerate INT8 cosine rounding on main by @wxyucs in #2969
  • fix(ci): trust Coverage workspace for Git verification by @wxyucs in #2965
  • fix(mci): keep maximal cliques whole and inline the filter bitmap by @LightWant in #2973
  • fix(pyramid): exclude reserved slots from label lookup by @wxyucs in #2966
  • fix(datacell): replace shared_mutex with atomic in FlattenInterface TotalCount by @LHT129 in #3006
  • fix: test HGraph MCI budgets per expansion by @wxyucs in #3034
  • fix: preserve RaBitQ raw binary tails on NEON by @wxyucs in #3035
  • fix: FlattenDataCell compilation fails when std::min receives an atomic count by @vsag-bot in #3053
  • fix(ivf): bound centroid scan routing memory by @shenguoquan in #3055
  • bug: check remaining duplicate build candidates by @jac0626 in #2891
  • fix(pyramid): surface ignored parallelism and document its semantics by @jac0626 in #2991
  • fix(api): handle serial GetDataByIds execution by @wellkilo in #2937
  • fix(hgraph): populate companion MCI stats in GetStats() when GetNumElements() == 0 by @thlurte in #3088
  • test(hgraph): keep analyzer assertions out of MCI workers by @LightWant in #3095
  • fix(pyramid): preserve reverse label lookup in immutable indexes by @dasurax in #3109
  • fix(simq): reject multi-query batch search instead of silently truncating by @LHT129 in #3081
  • test(pyramid): add regression test for root KNN duplicate ID return (#3036) by @thlurte in #3044
  • fix(hgraph): relax Tune source coverage by @jac0626 in #2719
  • fix(hgraph): keep a usable entry point for batch builds without upper levels by @LightWant in #3129
  • fix(hgraph): reject incomplete Tune sources after Add failure by @wxyucs in #3141
  • fix(pq): validate geometry in DeserializeImpl to prevent heap OOB write by @wxyucs in #3138
  • fix(build): avoid GCC sparse duplicate copy warning on main by @wxyucs in #3151
  • fix: stabilize existing-index AutoTune example construction by @wxyucs in #3156

New Features

  • feat(quantization): accelerate multibit rabitq encoding by @LightWant in #2446
  • feat(datacell): add bucket insert with explicit offset by @LHT129 in #2480
  • feat(io): add io_uring backend by @LHT129 in #2389
  • feat: add IVF bucket routing search mode by @LHT129 in #2495
  • feat(hgraph): dedup slot redirect for duplicate vector storage by @jac0626 in #2244
  • feat(ivf): add graph bucket searcher for large bucket approximate search by @LHT129 in #2494
  • feat: add HGraph MCI clique companion index by @LightWant in #2385
  • feat(io): add optional read cache for disk IO by @LHT129 in #2567
  • feat(sindi): integrate reasoning mechanism for search diagnostics by @LHT129 in #2405
  • feat: support custom batch query distances by @LHT129 in #2551
  • feat: expose split RaBitQ parameters for Pyramid by @LightWant in #2457
  • feat: add threshold filtering to knn search by @wxyucs in #2531
  • feat(ivf): support user-provided bucket_ids to bypass IVF bucket routing by @LHT129 in #2597
  • feat: support MRLE with split RaBitQ by @LightWant in #2560
  • feat: expose per-search distance evaluation statistics by @wxyucs in #2545
  • feat(ivf): support multi-query batch search with bucket_ids bypass by @LHT129 in #2639
  • feat: add AutoTune V1 orchestration by @jac0626 in #2569
  • feat: support pyramid raw vector retrieval by @jac0626 in #2503
  • feat(pyramid): support path-aware AnalyzeIndexBySearch by @jac0626 in #2668
  • feat(build): qualify third-party overrides by dependency pin by @wxyucs in #2679
  • feat(ivf): add explicit bucket graph rebuild API by @LHT129 in #2676
  • feat(ivf): support bucket-aligned precise codes by @jac0626 in #2520
  • feat(sindi): add filter callback limit by @Roxanne0321 in #2697
  • feat(pyramid): generalize build-cache feature so Pyramid is reusable by @LHT129 in #2375
  • feat: add SINDI V2 with term-first external IO by @Roxanne0321 in #2129
  • feat(pyramid): integrate reasoning mechanism for Pyramid index by @LHT129 in #2333
  • feat(hgraph): restore conjugate graph enhancement by @wxyucs in #2675
  • feat: add k-medoids clustering with data-point centers by @LHT129 in #2816
  • feat: support host filtering and streaming across SINDI modes by @Roxanne0321 in #2701
  • feat(pyramid): add fast multi-layer root routing by @jac0626 in #2717
  • feat: add date filtering to SINDI indexes by @Roxanne0321 in #2752
  • feat(ivf): add centroid scan routing switch by @LHT129 in #2832
  • feat(sindi): accept string host metadata by @Roxanne0321 in #2880
  • feat(attr): port ads attribute filtering capabilities to main by @LHT129 in #2909
  • feat(storage): add chunked serialization and parallel deserialization by @geerniman in #2748
  • feat(pyramid): support multiple paths per vector by @jac0626 in #2796
  • feat(search): add HGraph multi-query batch search support by @LHT129 in #2607
  • feat: add typed search statistics results by @LHT129 in #2897
  • feat: add timeout_ms support to SIMQ, SINDI, and SINDI V2 indexes by @LHT129 in #2919
  • feat: add parallel PiPNN graph construction by @jac0626 in #2858
  • feat: support fp16 reranking in sindi indexes by @Roxanne0321 in #2957
  • feat: add dynamic neighbor traversal for filtered HGraph search by @LightWant in #3000
  • feat(ivf): support RaBitQ x+y split storage and search by @LightWant in #2707
  • feat(pyramid): support duplicate-aware build cache by @LHT129 in #2948
  • feat: add SINDI publish timestamp filtering by @Roxanne0321 in #2981
  • feat: add SIMQ analyzer with doc vector count distribution stats by @LHT129 in #3011
  • feat(simq): support SetImmutable for concurrent read-only search by @LHT129 in #3023
  • add SearchSession for HGraph continue search by @LHT129 in #2921
  • feat: add local SQ4/SQ8 for HGraph L2, IP and cosine by @wxyucs in #3060
  • feat(reasoning): complete search diagnostics support by @LHT129 in #2846
  • feat: unify index analysis capabilities by @LHT129 in #2844
  • feat(ivf): parallelize custom distance search with late filtering by @LHT129 in #3122
  • feat(pyramid): support fp16 and bf16 dataset input by @LHT129 in #3126

Improvements

  • refactor: remove virtual destructor from BasicIO to restore CRTP zero-overhead by @LHT129 in #2399
  • perf: accelerate RaBitQ split IP search by @LightWant in #2444
  • perf(hgraph): reduce visited list memory by @jac0626 in #2449
  • perf(simq): batch rerank IO via MultiRead for async IO backend by @Sia-Sheerland in #2469
  • feat(ivf): abstract bucket searcher interface for extensibility by @LHT129 in #2461
  • chore: split bucket interface factory compilation by @wxyucs in #2458
  • perf(hgraph): reduce node lock memory overhead by @jac0626 in #2456
  • perf(sindi): precompute DMQ rerank query lookups by @LightWant in #2413
  • feat: add instance spec to init banner by @wxyucs in #2412
  • fix(ci): simplify docs sync lease handling by @wxyucs in #2521
  • feat: support multi-query batch distance calculation in CalDistanceById by @LHT129 in #2422
  • fix: reclaim BruteForce force-remove storage by @LHT129 in #2512
  • perf(sindi): optimize DMQ codebooks and packed ID decoding by @Roxanne0321 in #2523
  • perf(hgraph): optimize RaBitQ split construction by @LightWant in #2460
  • feat: unify distance-by-ID API naming with legacy compatibility by @wxyucs in #2532
  • fix(io): add RAII destructor to DirectIOObject and simplify AsyncIO ownership by @jfeng18 in #2280
  • ci: prefetch dependencies for host PR builds by @wxyucs in #2557
  • perf(pyramid): parallelize initial code encoding by @jac0626 in #2584
  • refactor(ivf): reuse shared graph search primitives by @LHT129 in #2602
  • perf(pyramid): reuse query computers in stats by @jac0626 in #2611
  • build: add experimental CRoaring amalgamation by @wxyucs in #2556
  • feat(sindi): implement value-aware term pruning by @Roxanne0321 in #2542
  • perf: avoid copying DiskANN streams for size checks by @Hastyshell in #2622
  • fix: bound DiskANN closest-center block memory by @Hastyshell in #2629
  • feat(pyramid): support root graph hops limit by @dasurax in #2642
  • fix: align MCI build and search semantics by @LightWant in #2628
  • improve(pyramid): use RAII guard for VisitedList pool handles by @jfeng18 in #2282
  • refactor: remove Pyramid raw vector dependency for split codes by @LightWant in #2643
  • test: make filtered range recall diagnostic by @jac0626 in #2655
  • refactor!: remove HNSW and DiskANN; legacy indexes become unsupported by @wxyucs in #2624
  • improve(ivf): let train_sample_count scale with buckets_count by @LHT129 in #2671
  • fix: support debug builds with GCC 15 by @wxyucs in #2681
  • ci: publish container images to GHCR by @wxyucs in #2690
  • refactor(datacell): extract fixed layout storage by @LHT129 in #2725
  • feat(simq): multi-threaded build/search, token quantization, flat-array coarse search by @Sia-Sheerland in #2713
  • perf(io): batch cache misses in multiread by @LHT129 in #2636
  • perf: eliminate TQ transform allocations by @LHT129 in #2703
  • refactor(datacell): extract variable record layouts by @LHT129 in #2737
  • ci: publish build performance metrics by @wxyucs in #2781
  • perf(build): separate cached Release builds by @wxyucs in #2782
  • perf(build): narrow datacell Ninja job pool by @wxyucs in #2784
  • refactor(build): reuse library object files by @wxyucs in #2785
  • chore: enforce CamelCase struct names by @wxyucs in #2764
  • chore: remove mock implementation by @wxyucs in #2787
  • feat(pyramid): support paths in GetDataByIdsWithFlag by @jac0626 in #2763
  • ci: expose OpenBLAS build failure logs by @jac0626 in #2800
  • refactor(parameters): replace template-based index mappings by @LHT129 in #2730
  • perf(hgraph): optimize RaBitQ split search hot path by @LightWant in #2813
  • refactor(datacell): use FixedLayout for graph records by @LHT129 in #2820
  • refactor(ivf): split implementation by responsibility by @LHT129 in #2804
  • chore: organize repository labels by @wxyucs in #2839
  • perf(ivf): keep routing statistics parsing out of ClassifyDatas reduce lock by @LHT129 in #2838
  • chore: align issue templates with label taxonomy by @wxyucs in #2848
  • chore: remove stale issue template labels by @wxyucs in #2852
  • chore: remove legacy labels from manifest by @wxyucs in #2853
  • refactor(io): introduce composable high-performance storage architecture by @LHT129 in #2793
  • perf(hgraph): fuse mutable RaBitQ split storage and search for x=1..4 by @LightWant in #2596
  • build: prefer Ninja with Unix Makefiles fallback by @wxyucs in #2854
  • perf(build): reduce unit-test compile time with scoped PCH by @wxyucs in #2855
  • perf(build): reduce flatten factory compile and lint hotspots by @wxyucs in #2857
  • ci: use generic OpenBLAS baseline in x86 wheel and daily builds by @jac0626 in #2819
  • perf(sindi): remove approximate evaluation tracking by @Roxanne0321 in #2873
  • refactor(datacell): replace direct BasicIO access with ByteRangeLayout in DiskSindiTermDataCell by @LHT129 in #2850
  • ci: make C++ coverage repository-scoped by @wxyucs in #2887
  • perf(pyramid): accelerate build cache restoration by @LHT129 in #2900
  • feat(sindi): support missing date labels by @Roxanne0321 in #2935
  • build: add independently runnable module tests by @wxyucs in #2856
  • chore(build): upgrade OpenBLAS to 0.3.34 by @wxyucs in #2952
  • ci: add scoped pre-merge SIMD coverage protection by @wxyucs in #2914
  • ci: make dependency build metrics trustworthy by @wxyucs in #2899
  • perf(pyramid): accelerate RaBitQ split construction with scalar SIMD by @LightWant in #2959
  • build(gpu): add an optional CUDA backend target, off by default by @seuclkzsd in #2860
  • ci: separate PR ASan builds from performance benchmarks by @LHT129 in #2955
  • feat(sindi_v2): add filter callback limit by @Roxanne0321 in #2982
  • feat(pyramid): add extra_info support for build, search, and serialization by @LHT129 in #2994
  • feat: support fp16 and bf16 dataset input for SINDI and SINDIV2 indexes by @LHT129 in #3008
  • chore: support dependency URL files in dist packaging by @wxyucs in #3033
  • feat(ivf): build IVF indexes on the CUDA backend by @seuclkzsd in #3022
  • feat(hgraph): support physical removal and incremental maintenance for MCI by @LightWant in #3012
  • perf(mci): parallelize and plan incremental Add batches lock free by @LightWant in #3063
  • feat(ivf): expose kmeans training iteration count as a build parameter by @LHT129 in #3042
  • feat: advertise SUPPORT_BATCH_SEARCH for HGraph and IVF by @LHT129 in #3047
  • feat(simq): build partitions with native quantized data by @LHT129 in #2996
  • perf(pipnn): add opt-in adaptive alpha pruning by @LightWant in #3049
  • feat(sindi): support build-time term pruning by @Roxanne0321 in #3071
  • perf(hgraph): share vector storage for PiPNN duplicates by @LightWant in #3097
  • perf(hgraph): add opt-in adaptive NSW neighbor pruning by @LightWant in #2933
  • refactor(index): centralize candidate graph construction with GraphBuildHelper by @LHT129 in #3003
  • fix(pyramid): honor per-request search parallelism by @dasurax in #3124
  • perf: cache SINDI v2 disk postings with a bounded LRU by @Roxanne0321 in #3067
  • perf(pipnn): reduce graph construction overhead by @jac0626 in #3117
  • chore(ci): pause scheduled daily tests pending timeout investigation by @wxyucs in #3134
  • ci: preinstall tooling for non-root x86 CI jobs by @wxyucs in #3147
  • [skip ci] ci: add maintainer allowlist for team routing by @wxyucs in #3146
  • [skip ci] ci: add main-branch team routing configuration by @wxyucs in #3150

Other Changes

  • docs: add Chinese SINDI introduction blog by @Roxanne0321 in #2440
  • docs: add SIMQ index documentation for en and zh by @Sia-Sheerland in #2441
  • docs: add new index integration checklist by @wxyucs in #2426
  • docs: add VSAG 1.0 release notes by @wxyucs in #2453
  • fix(pyramid): promote flat nodes at size threshold by @jac0626 in #2464
  • docs: document features added since v0.18 by @wxyucs in #2539
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2262
  • feat: configure resize batch for HGraph and BruteForce by @LHT129 in #2541
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2552
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2632
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2649
  • docs: clarify agent contribution guidelines by @wxyucs in #2682
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2723
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2741
  • docs(contributor): update contributors list by @wxyucs in #2742
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2757
  • docs: align dense index lifecycle documentation by @LHT129 in #2762
  • refactor(parameters): remove legacy mapping utilities by @LHT129 in #2822
  • docs: add C++ coding guide by @wxyucs in #2823
  • feat(io): add opt-in prefetch hints by @LHT129 in #2886
  • docs: clarify SIMQ full name by @wxyucs in #2910
  • docs: fix site links and stale guidance by @wxyucs in #2882
  • [skip ci] docs: clarify generated version header reference by @wxyucs in #2950
  • [skip ci] docs: update EnhanceGraph publication references to TKDE by @wxyucs in #2928
  • [skip ci] docs: add v1.0.1 release notes by @wxyucs in #2997
  • docs(contributor): contributors readme action update by @github-actions[bot] in #2960
  • perf(hgraph): speed up the fused RaBitQ incremental build (issue #3007) by @LightWant in #3018
  • docs(contributor): contributors readme action update by @github-actions[bot] in #3085
  • docs(contributor): contributors readme action update by @github-actions[bot] in #3115
  • docs(contributor): contributors readme action update by @github-actions[bot] in #3130

New Contributors

Full Changelog: v1.0.0...v1.1.0