v0.10.16

Bug fixes

• fix: use 1/ln(m) as the mL value for better performance by @BubbleCal in #2239
• fix: various fixes to ensure that the v2 writer is used by the ray sink if requested by @westonpace in #2248
• fix: bugs in list codec when dealing with a list of empty lists or multiple lists in a single page by @westonpace in #2222
• fix: creating IVF_HNSW index fails with cosine metric type by @BubbleCal in #2235
• fix: build & search causes panic with empty partition for IVF_HNSW by @BubbleCal in #2172
• fix: clippy on main by @chebbyChefNEQ in #2254
• fix: copy arrays when placing them in the v2 writer's accumulation queue by @westonpace in #2249
• fix: out of bound panics in take rows by @chebbyChefNEQ in #2259
• fix: allow for all-null pages in btree by @westonpace in #2245

New Features

• feat: add use_experimental_writer to write_fragments API by @westonpace in #2226
• feat: use v2 writer api in ray data sink by @eddyxu in #2231
• feat(java): add LanceOperation, commitAppend for batch write by @LuQQiu in #2207
• feat(python): expose FragmentMetadata attrs and compaction repr by @wjones127 in #2191
• feat: update arrow to 51, datafusion to 37 by @westonpace in #2240
• feat: improvement of Ray sink API by @eddyxu in #2237
• feat: forbid creating index if num_sub_vectors doesn't divide dim by @BubbleCal in #2234
• feat: add create_index to vector index extensions by @chebbyChefNEQ in #2250
• feat: support delta merge for IVF_HNSW_SQ by @BubbleCal in #2132
• feat(java): add version latestVersion api by @LuQQiu in #2238
• feat: raise GCS object size limit to 2.5TB by @wjones127 in #2261

Performance Improvements

• perf: concurrent building for HNSW graph by @BubbleCal in #2210
• perf: v2 fsl decode perf fix and some benchmarking utilities by @westonpace in #2214
• perf: avoid copying of creating memory dist calculator by @BubbleCal in #2219
• perf: keep less edges in non-base levels to improve greedy search performance by @BubbleCal in #2244

Other changes

• chore: remove --preview from ruff invocations by @westonpace in #2220
• chore: fix formatting in test_schema.py by @westonpace in #2227
• chore: bump to ruff 0.4.1 by @eddyxu in #2229
• chore: bump minimal python version to 39 by @eddyxu in #2230
• chore: use VectorStorage in HNSWBuilder by @chebbyChefNEQ in #2242
• test: test write methods against more varied layouts by @wjones127 in #2233

Full Changelog: v0.10.15...v0.10.16