Skip to content

v5.0.0-beta.1

Pre-release
Pre-release

Choose a tag to compare

@lance-community lance-community released this 27 Mar 22:47
· 1653 commits to main since this release

What's Changed

Breaking Changes 馃洜

  • refactor!: remove staging from distributed vector indexing by @Xuanwo in #6269
  • refactor: move DatasetIndexExt out of lance-index by @Xuanwo in #6280
  • feat!: support sampling selected fragments by @Xuanwo in #6294

New Features 馃帀

  • feat(java): add non-blocking AsyncScanner with CompletableFuture API by @beinan in #6102
  • feat(DirectoryNamespace): support index and transaction related operations by @zhangyue19921010 in #6196
  • feat: bounding source fragments for compaction execution by @hamersaw in #6232
  • fix: filter out detached versions when scanning manifests by @jackye1995 in #6245
  • feat: allow setting transaction properties in various operations by @jackye1995 in #6246
  • feat: add OpenDAL Azdls backend for abfss:// with use_opendal flag by @burlacio in #6256
  • feat: add aimd throttled object store by @westonpace in #6266
  • feat: clarify logical indices and physical index segments by @Xuanwo in #6270
  • feat: support stop-word gaps in phrase queries by @BubbleCal in #6277
  • feat: move rate limiting to the object store by @westonpace in #6293
  • feat: support non-shared centroid vector index builds by @Xuanwo in #6296
  • feat: add a fast dataset version ID API by @BubbleCal in #6303
  • feat(python): add storage_options to IvfModel and PqModel save/load by @hushengquan in #6312

Bug Fixes 馃悰

  • fix: handle list-level NULLs in NOT filters by @fenfeng9 in #6044
  • fix: like queries with a prefix should be accelerated by btree and zonemap by @jackye1995 in #6188
  • fix: respect the old data filter on inverted index by @westonpace in #6216
  • fix: 2.1/2.2 panic when a list column had small values and many empty values by @westonpace in #6234
  • fix: resolve_latest_location converts errors to not_found unconditionally by @wkalt in #6248
  • fix: return errors for unsupported fixed-size-list child types by @myandpr in #6253
  • fix: adding namespace support to java SDK CommitBuilder from dataset by @hamersaw in #6257
  • fix: pass dataset_options to SafeLanceDataset in worker processes by @eddyxu in #6278
  • fix: support hamming distance in IndicesBuilder by @jmhsieh in #6295
  • fix(namespace): support nested types in convert_json_arrow_type by @jiaoew1991 in #6300
  • fix: restore namespace build after DatasetIndexExt move by @Xuanwo in #6302
  • fix: multiple improvements for gh workflows by @esteban in #6306

Documentation 馃摎

  • docs: shorten core major release vote window by @Xuanwo in #6154

Performance Improvements 馃殌

  • perf: add benchmark for distributed vector merge finalization by @Xuanwo in #6176
  • perf: new layout for positions and new algo for phrase query by @BubbleCal in #6203
  • perf: batched WAND and new WAND structure, ~50% faster by @BubbleCal in #6241
  • perf: optimize stable row_id index build from O(rows) to O(fragments) by @jiaoew1991 in #6310

Other Changes

  • refactor: distributed vector segment build by @Xuanwo in #6220

Full Changelog: release-root/5.0.0-beta.N...v5.0.0-beta.1