v5.0.0-beta.1
Pre-release
Pre-release
·
1653 commits
to main
since this release
What's Changed
Breaking Changes 馃洜
- refactor!: remove staging from distributed vector indexing by @Xuanwo in #6269
- refactor: move DatasetIndexExt out of lance-index by @Xuanwo in #6280
- feat!: support sampling selected fragments by @Xuanwo in #6294
New Features 馃帀
- feat(java): add non-blocking AsyncScanner with CompletableFuture API by @beinan in #6102
- feat(DirectoryNamespace): support index and transaction related operations by @zhangyue19921010 in #6196
- feat: bounding source fragments for compaction execution by @hamersaw in #6232
- fix: filter out detached versions when scanning manifests by @jackye1995 in #6245
- feat: allow setting transaction properties in various operations by @jackye1995 in #6246
- feat: add OpenDAL Azdls backend for abfss:// with use_opendal flag by @burlacio in #6256
- feat: add aimd throttled object store by @westonpace in #6266
- feat: clarify logical indices and physical index segments by @Xuanwo in #6270
- feat: support stop-word gaps in phrase queries by @BubbleCal in #6277
- feat: move rate limiting to the object store by @westonpace in #6293
- feat: support non-shared centroid vector index builds by @Xuanwo in #6296
- feat: add a fast dataset version ID API by @BubbleCal in #6303
- feat(python): add storage_options to IvfModel and PqModel save/load by @hushengquan in #6312
Bug Fixes 馃悰
- fix: handle list-level NULLs in NOT filters by @fenfeng9 in #6044
- fix: like queries with a prefix should be accelerated by btree and zonemap by @jackye1995 in #6188
- fix: respect the old data filter on inverted index by @westonpace in #6216
- fix: 2.1/2.2 panic when a list column had small values and many empty values by @westonpace in #6234
- fix: resolve_latest_location converts errors to not_found unconditionally by @wkalt in #6248
- fix: return errors for unsupported fixed-size-list child types by @myandpr in #6253
- fix: adding namespace support to java SDK CommitBuilder from dataset by @hamersaw in #6257
- fix: pass dataset_options to SafeLanceDataset in worker processes by @eddyxu in #6278
- fix: support hamming distance in IndicesBuilder by @jmhsieh in #6295
- fix(namespace): support nested types in convert_json_arrow_type by @jiaoew1991 in #6300
- fix: restore namespace build after DatasetIndexExt move by @Xuanwo in #6302
- fix: multiple improvements for gh workflows by @esteban in #6306
Documentation 馃摎
Performance Improvements 馃殌
- perf: add benchmark for distributed vector merge finalization by @Xuanwo in #6176
- perf: new layout for positions and new algo for phrase query by @BubbleCal in #6203
- perf: batched WAND and new WAND structure, ~50% faster by @BubbleCal in #6241
- perf: optimize stable row_id index build from O(rows) to O(fragments) by @jiaoew1991 in #6310
Other Changes
Full Changelog: release-root/5.0.0-beta.N...v5.0.0-beta.1