v14.0.0-beta.16

What's Changed

Breaking Changes 🛠

• feat(index): change btree scalar index to row-address domain by @westonpace in #9481
• perf(mem_wal): optimize Blob v2 writes and promotion by @jackye1995 in #9717
• feat(cleanup): add configurable delete concurrency by @majin1102 in #9774

New Features 🎉

• feat(python): expose the fragment ids of a dataset by @zhangyue19921010 in #9592
• feat: record row-address moves for deferred compactions on stable-row-ID tables by @brendanclement in #9611
• feat(dataset): support stable spilled row lineage by @BubbleCal in #9802
• feat(io): make file-read coalescing configurable by @BubbleCal in #9833
• feat: synchronous FilteredReadExec encoding given a resolved table identifier by @LuQQiu in #9919

Bug Fixes 🐛

• fix(mem_wal): full-text search across columns added or renamed since a source was written by @xuanyu-z in #9740
• fix: keep scan order when the optimizer adds parallelism by @vivek-bharathan in #9819
• fix(mem_wal): skip already-flushed generations when replaying the WAL by @xuanyu-z in #9840
• fix(mem_wal): read each memtable's visible batches once per LSM query by @xuanyu-z in #9841
• fix(index): write zone map seeds when a seeded column gets a new data file by @LuQQiu in #9868
• fix: count rows selected by row id or row address by @vivek-bharathan in #9907
• fix(encoding): do not materialize the constant scalar into null slots by @dsingh-exa in #9909
• fix(encoding): bounds-check the item payload in VariableFullZipDecoder::unzip by @dsingh-exa in #9910
• fix(io): call multipart complete once and retry it at the HTTP layer by @dsingh-exa in #9913
• fix(lance-jsonb): escape object keys when displaying Value by @westonpace in #9951
• fix(mem_wal): keep field ids on prepared blob v2 columns by @xuanyu-z in #9957
• fix(mem_wal): emit an LSM scan's sources in a fixed order by @hamersaw in #9983

Documentation 📚

• docs(format): stabilize spilled row lineage by @BubbleCal in #9886

Performance Improvements 🚀

• perf: prune CDF fragments using row-version metadata by @Abhisheklearn12 in #9409
• perf(index): bound merge_segments prefetch to one scan's byte budget by @amunra in #9761
• perf(dataset): serve read_version_transaction from the held or cached manifest by @dsingh-exa in #9915
• perf(python): read the version id directly when committing a transaction by @dsingh-exa in #9916
• perf(fragment): read the version id directly when writing deletion files by @dsingh-exa in #9917
• perf(encoding): write zstd output into reserved capacity instead of zero-filling by @dsingh-exa in #9918
• perf(index): read the manifest version without summarizing every fragment by @dshepelev15 in #9942
• perf(scanner): serve null-check filters from a narrow leaf by @wkalt in #9948
• perf(merge_insert): evaluate target-column conditions on the rows they gate by @wkalt in #9949

Other Changes

• refactor: split dataset/blob.rs into submodules by stage by @wjones127 in #9989

Full Changelog: v14.0.0-beta.15...v14.0.0-beta.16