v0.27.0
What's Changed
Breaking Changes 馃洜
New Features 馃帀
- feat: add public accessors for lance scan fields by @wkalt in #3739
- feat: store data file size in manifest by @wjones127 in #3750
- feat: expose public accessor for LanceScanExec config by @wkalt in #3757
- feat: fix various spans by @wjones127 in #3738
- feat(build): add make clean target for Python temporary files by @Jay-ju in #3756
- feat: add tracing spans to ScanExec and TakeExec by @wjones127 in #3766
- feat: make it possible to filter on large_binary by @westonpace in #3797
- feat: split large reads into smaller parallel reads by @westonpace in #3801
- feat: introduce new FilteredReadExec which will replace the initial scan/take by @westonpace in #3782
- feat: support LANCE_FILE_WRITER_MAX_PAGE_BYTES env by @yanghua in #3815
Bug Fixes 馃悰
- fix: prevent infinite loop of writing manifest by @wjones127 in #3760
- fix: revert IO optimizations for now until we can test more by @wjones127 in #3763
- fix: don't lose rows during bitmap remap if rows not being remapped by @westonpace in #3765
- fix: flat scan frags not covered by scalar index when doing prefilter search by @westonpace in #3768
- fix: partition not used yet panic by @BubbleCal in #3769
- perf: reduce retry thrashing in merge_insert by @wjones127 in #3780
- fix: fix mapdataset batch getitems by @Jay-ju in #3783
- fix: revert tracing change from #3766 by @wjones127 in #3800
- fix: limit the batch size to 64 for indexing multivectors by @BubbleCal in #3810
- fix: logic bug in RowIdTreeMap union by @westonpace in #3811
- fix: handle bitmap index overflow by implementing multi-batch read&write #3515 by @HaochengLIU in #3789
- fix: fix python tracing by @westonpace in #3807
Documentation 馃摎
- docs: how to set blob metadata by @eddyxu in #3759
- docs:clarify instructions on where to run maturin develop by @HaochengLIU in #3767
- docs: fix typos by @omahs in #3803
Performance Improvements 馃殌
- perf: read tiny files in 1 IOP by @wjones127 in #3752
- perf: optimize load_indices by @wjones127 in #3762
- perf: bring back io optimizations by @wjones127 in #3764
New Contributors
- @HaochengLIU made their first contribution in #3767
- @omahs made their first contribution in #3803
Full Changelog: v0.26.1...v0.27.0