Repository navigation
v15.0.0-beta.20
Pre-release
Pre-release
·
70 commits
to main
since this release
What's Changed
Breaking Changes 🛠
- feat(index)!: change ngram, rtree, and minhash lsh scalar indexes to row-address domain by @westonpace in #9858
- perf(encoding): push down MiniBlock range decode and O(1) bitpack take by @Xuanwo in #10909
- fix(namespace): preserve timestamp and duration precision by @Xuanwo in #11059
Critical Fixes ‼️
- fix(dataset): honor target stores in shallow clones by @app/lance-gatefixer in #10942
New Features 🎉
- feat(session): allow resizing cache capacity at runtime by @hello-peter-tang in #10657
- feat(index): support independent covering fields by @Ali2Arslan in #10662
- feat(python): add session metadata cache to file readers by @hello-peter-tang in #10713
- feat: log index build memory as trace events by @wombatu-kun in #10754
- feat: report memory held by I/O schedulers, DataFusion pools and MemWAL by @wombatu-kun in #10760
- feat(python): support bounded vector index consolidation by @zhangyue19921010 in #10762
- feat(index): route whole predicates to multi-column index planners by @Ali2Arslan in #10768
- feat: add WARP multivector search by @jzheng106 in #10927
- feat(format): add the listelement scalar index specification (experimental) by @sarwarbhuiyan in #10949
- feat: add the experimental listelement scalar index over list elements by @sarwarbhuiyan in #10950
- feat(blob): add service URI policy and External cache hooks by @Xuanwo in #10989
- feat: report the peak memory reservation of each execution by @wombatu-kun in #11037
Bug Fixes 🐛
- fix(dataset): reject branch names that exceed storage limits by @app/lance-gatefixer in #10734
- fix: persist nested view arrays in Arrow Variant columns by @beinan in #10739
- fix(namespace): publish S3 markers with conditional PUT by @app/lance-gatefixer in #10764
- fix(dataset): reject incompatible UDF checkpoints by @jackylee-ch in #10771
- fix(python): train ANN test fixture on all rows by @app/lance-gatefixer in #10785
- fix(index): don't cache the tagged FRI listing before vector details are inferred by @zehiko in #10791
- fix(index): reject incompatible SQ models during segment merge by @ddupg in #10797
- fix(encoding): decode empty variable packed structs by @BubbleCal in #10830
- fix(index): avoid false consolidation conflicts after refreshing a dataset by @ddupg in #10888
- fix(java): release row address lookup lock before I/O by @Xuanwo in #10931
- fix(file): restore Blob v2 descriptor children in the raw file reader by @ump45nose in #10946
- fix(mem-wal): reject dictionary overflow before accepting writes by @app/lance-gatefixer in #11001
- fix(fts): consolidate small partitions during incremental updates by @BubbleCal in #11011
- fix: reject a merge or update whose right side has only the join key by @LuciferYang in #11030
- fix: reject a replacement data file whose column types differ from the dataset by @LuciferYang in #11031
- fix: apply chained field metadata updates for one field in order by @LuciferYang in #11033
- fix(index): classify invalid search parameters as input errors by @jackye1995 in #11034
- fix(test): serialize allocation group registration by @Xuanwo in #11044
Documentation 📚
- docs: align the FTS format spec with InvertedIndexParams by @hello-peter-tang in #9786
- docs: say approx_mode also affects HNSW searches that exclude rows by @LuciferYang in #10751
- docs(encoding): document dictionary gather buffer safety by @Xuanwo in #10980
Performance Improvements 🚀
- perf(vector): support adaptive nprobes in shared-scan batch IVF search by @sezruby in #10443
- perf(index): compile IVF search/build scheduling once, not per S×Q by @wjones127 in #10727
- perf(index): overlap RTree page reads in prewarm and search_bbox by @LuciferYang in #10748
- perf(index): stop LabelList has-all-labels search once the intersection is empty by @LuciferYang in #10749
- perf(index): size BloomFilter zones without serializing and skip the copy on remapped load by @LuciferYang in #10750
- perf(scan): resolve SQL filter column types against the scanner's schema by @zehiko in #10867
- perf(encoding): optimize FSST symbol-table reuse and buffer handling by @Xuanwo in #10907
- perf(encoding): accelerate dictionary interning, sampling, and decode gather by @Xuanwo in #10911
- perf(encoding): compute block stats lazily and accelerate repdef validity paths by @Xuanwo in #10958
- perf(dataset): read manifests concurrently in Dataset::versions by @jackylee-ch in #10966
- perf(encoding): add AVX-512/AVX2 FastLanes bitpacking dispatch and trim chunk allocations by @Xuanwo in #10967
- perf(encoding): accelerate ALP, RLE, BinaryMiniBlock, AxisDelta, and FullZip paths by @Xuanwo in #10972
- perf(fts): skip filtered partitions before loading postings by @BubbleCal in #10988
- perf: binary-search the deletions when resolving fragment take offsets by @LuciferYang in #11028
- perf(fts): share the jieba dictionary between tokenizer clones by @ethqunzhong in #11056
- perf(index): overlap shared RaBitQ prewarm planes by @BubbleCal in #11058
Full Changelog: v15.0.0-beta.19...v15.0.0-beta.20