v0.25.0

What's Changed

Breaking Changes 🛠

• feat!: create index in v3 version by default by @BubbleCal in #3477

New Features 🎉

• feat: write_dataset from pylist and pydict by @eddyxu in #3527
• feat: rework how we train ngram indices for better performance by @westonpace in #3518
• feat: expose make_deletions_null to python as include_deleted_rows by @westonpace in #3533
• feat: add project transaction operation for pylance sdk by @SaintBacchus in #3538
• feat: don't log span info by @westonpace in #3547
• feat: support retrain index and incremental kmeans by @BubbleCal in #3489
• feat: add tracing events for I/O, index loading, and plan execution by @westonpace in #3575
• feat: make it possible to get the field ids from a lance_schema by @westonpace in #3568
• feat(python): add warning about fork by @wjones127 in #3584
• feat: add tracing to cleanup by @wjones127 in #3585
• feat: add JNI bindings for the file reader/writer by @westonpace in #3588

Bug Fixes 🐛

• fix(java): java version is out of sync with rust and python by @yanghua in #3546
• fix: indexing time in unit tests is much slower than before by @BubbleCal in #3561
• fix: the PQ codes corrupted after remapping by @BubbleCal in #3573
• fix: remove some expensive debug impls by @westonpace in #3576
• fix: reintroduce TakeExec.dataset method by @wkalt in #3577
• fix(android): compilation error on android by @TD-Sky in #3555
• fix: work around deranged breaking change not labeled as such by @westonpace in #3591

Documentation 📚

• docs: fix read_and_write example by @eddyxu in #3521
• docs: organize contents into sections by @eddyxu in #3528
• docs: update ray integration and move schema evolution doc to a separate doc by @eddyxu in #3530
• docs: fix checklinks by @eddyxu in #3532
• docs: add example of Dataset.insert by @eddyxu in #3534
• docs: update README to include new table format and format v2 blogs by @eddyxu in #3535
• docs: enable merge insert doctest by @eddyxu in #3542
• docs: raw distributed write by @eddyxu in #3548
• docs: how to use tags by @eddyxu in #3562

Performance Improvements 🚀

• perf: coalesce continuous indices into ranges if possible by @niyue in #3513
• perf: improve v3 indexing perf by @BubbleCal in #3525
• perf: improve 4bit PQ performance by @BubbleCal in #3557

Other Changes

• refactor: rework how take handles parallelism by @westonpace in #3543
• refactor(rust): fix build_predicate misleading row_ids replace to row_addrs by @yanghua in #3551

New Contributors

• @TD-Sky made their first contribution in #3555

Full Changelog: v0.24.1...v0.25.0