v3.0.0-beta.2

What's Changed

Critical Fixes ‼️

• fix: deduplicate row addresses in take to prevent panic by @wjones127 in #5881
• fix: fts flat search drops rows when avg_doc_length < 1.0 by @wjones127 in #5897

New Features 🎉

• feat: add RLE support for block by @yingjianwu98 in #4937
• feat: dictionary index always32 bits by @yingjianwu98 in #5011
• feat: abort dictionary encode if not useful by @yingjianwu98 in #5055
• feat(cdf): cdf support upsert for views by @zhangyue19921010 in #5369
• feat(compaction): binary copy capability for compaction by @zhangyue19921010 in #5434
• feat: add alter column nullable to non-nullable support by @Xuanwo in #5589
• feat: evolute all_null_layout to constant layout by @Xuanwo in #5641
• feat(java): support building vector index distributively by @majin1102 in #5664
• feat(rust): add datafusion catalog_provider through namespace by @majin1102 in #5686
• feat: support List and Struct type for KeyValue in inserted_rows.rs by @wojiaodoubao in #5713
• feat: support tencent cos by @ztorchan in #5740
• feat: add Lance-HF docs to lance.org/integrations/huggingface/ by @prrao87 in #5748
• feat(python): support namespace for tensorflow by @yuqi1129 in #5750
• feat(java): support json extraction by scanning by @majin1102 in #5770
• feat: expose blob handling APIs to python by @Xuanwo in #5790
• feat: add blob handling support for fragment by @Xuanwo in #5801
• feat: add plan/execute separation to FilteredReadExec by @LuQQiu in #5843

Bug Fixes 🐛

• fix: support system columns in dataset.take* operations by @hamersaw in #5722
• fix: skip missing indices in compaction rewrite by @AndreaBozzo in #5739
• fix(lance-linalg): check fp16kernels feature before arch-specific code by @durch in #5747
• refactor: align blob behavior that write via file format version, read via layout by @Xuanwo in #5752
• fix: fix deletion when using file-object-store:// by @cmccabe in #5760
• fix: remove unreasonable nullable check for data types in hash_joiner during merge operation by @zhangyue19921010 in #5784
• fix: allow unused_unsafe for __cpuid to support both stable and nightly by @jackye1995 in #5793
• fix: set JUnit dependency as test scope by @bryanck in #5815
• fix: fix remap so that it handles deletions correctly by @westonpace in #5828
• fix: inconsistent transposed pq code and metadata when build ivf_pq index distributedly by @yanghua in #5834
• fix(java): panic when reading CreateIndex transaction by @majin1102 in #5853
• fix: fix mini-block dictionary bitpacking panic by @Xuanwo in #5860
• fix: fix boolean inline constant decoding by @Xuanwo in #5862
• fix: open additional storage options provider related apis in lance dataset by @jackye1995 in #5869
• fix: flaky test test_ann_prefilter for HNSW by @BubbleCal in #5870
• fix(java): init allocator for new dataset when checkout branch/tag by @fangbo in #5876
• fix: avoid panic when repdef serializes empty offsets by @fenfeng9 in #5890
• fix: avoid bitmap range panic on inverted bounds by @fenfeng9 in #5893
• fix: split index_statistics to reduce rustc query depth by @Xuanwo in #5894

Documentation 📚

• docs: fix issues in HF integration docs by @prrao87 in #5778
• docs: fix MkDocs protobuf reference for ConstantLayout by @Xuanwo in #5833
• docs: add array type support by @XuQianJin-Stars in #5884

Performance Improvements 🚀

• perf: add vector throughput benchmark by @westonpace in #5644
• perf: add a full text search benchmark by @westonpace in #5665
• perf: don't concat the batches for writing posting lists by @BubbleCal in #5769
• perf: add a lightweight scheduler implementation by @westonpace in #5773
• perf: use cpu pool to process all posting lists by @BubbleCal in #5780
• perf: calculate cardinality lazily by @Xuanwo in #5783
• perf: replace flatmap in build_distance_table by @wkalt in #5898

Other Changes

• refactor: change reader's get_range result to be a static future by @westonpace in #5755
• refactor(python): migrate torch.jit.script to torch.compile by @wjones127 in #5759
• test: fix tests broken by pandas 3 release by @westonpace in #5786

Full Changelog: release-root/3.0.0-beta.N...v3.0.0-beta.2