Changes
⚠️ Breaks
- aggregate fns to have grouped aggregate kernels for sum and count (#8314) @onursatici
- Remove
lenparameter fromScalarFnArray::try_new(#8378) @connortsui20 - Remove all TurboQuant stuff (#8377) @connortsui20
- Fix Validity::mask_eq semantics for mixed variants (#8334) @joseph-isaacs
- Rename Validity::no_nulls to definitely_no_nulls and add execute_no_nulls (#8333) @joseph-isaacs
- Pass execution context through list view rebuild (#8274) @connortsui20
- Initialize LazyBitBufferBuilder from Mask if available, when appending masks pass them by reference (#8221) @robert3005
- layout reader ctx (#8037) @onursatici
🚧 Deprecation
- Refactor validity checks to require explicit ExecutionCtx (#8273) @joseph-isaacs
- Deprecate vortex-array public APIs that use the hidden LEGACY_SESSION (#8269) @joseph-isaacs
✨ Features
- feat(vortex-geo): native Point extension type and GeoDistance scalar function (#8372) @HarukiMoriarty
- Add arrow import/export for vortex-json (#8339) @AdamGS
- WKB import/export VTables to Arrow (#8325) @a10y
- feat[gpu]: export as Arrow device array binary (#8320) @0ax1
- feat[gpu]: sliced validity in Arrow device export (#8318) @0ax1
- add vortex-geo crate and WKB extension type (#7722) @a10y
- Expose Merge expression in vortex-jni (#8319) @robert3005
- feat(vortex-array): add Interleave array encoding (#8277) @joseph-isaacs
- feat: add cuDF C FFI (#8236) @0ax1
- feat: export dictionary arrow device arrays (#8251) @0ax1
- Add zstd compression schemes for binary data (#8304) @AdamGS
- Upgrade DataFusion to 54 (#8044) @AdamGS
- byte_length() expression (#8298) @myrrc
- feat(vortex-row): row-oriented byte encoder (size + encode passes) (#8253) @joseph-isaacs
- Support pack expressions through JNI (#8266) @AdamGS
- feat: very wide column benchmark (#8252) @joseph-isaacs
- Support ParquetVariant through JNI (#8129) @AdamGS
- Introduce a new JSON extension type in
vortex-json(#8215) @AdamGS - feat[gpu]: arrow device array decimal export (#8155) @0ax1
- feat[gpu]: arrow device array list view export (#8219) @0ax1
- Add RoaringBitmap support to vortex-jni bindings (#8220) @robert3005
- Implement ZipKernel for ListViewArray (#8218) @robert3005
- move towards
BitBuffer/{Mut}Viewover owned bit buffers (#8208) @joseph-isaacs
🚀 Performance
- perf: branchless primitive zip kernel (#8270) @joseph-isaacs
- perf[gpu]: coalesce streaming kernels (#8357) @0ax1
- perf[mask]: check AllTrue/AllFalse in eq (#8354) @joseph-isaacs
- BitBuffer binary operations support in place updates to lhs (#8162) @robert3005
- Optimize slice for dict and minor changes in other arrays (#8321) @AdamGS
- perf: remove unneeded scalar_at and valid_at and use arrays instead (#8307) @joseph-isaacs
- Allow caching LayoutReader in VortexFile (#8244) @myrrc
- perf: branchless boolean zip kernel (#8275) @joseph-isaacs
- Initial impl for sort pushdown in DataFusion FileSource implementation (#8235) @AdamGS
- Optimize dict array validation to use true_count() (#8263) @joseph-isaacs
- Add #[inline] hints to hot path functions (#8260) @joseph-isaacs
- perf(fastlanes): fuse bit-packed compare into a transposed mask + untranspose (#8239) @joseph-isaacs
- case_when with is_null -> fill_null (#8245) @onursatici
- perf: optimize varbinview construction (#8146) @joseph-isaacs
- ViewedLayoutChildren child layout cache (#8234) @myrrc
- Don't recalculate chunk offsets for ChunkedReader (#8231) @myrrc
- Share FSST symbol table state between executed variants of arrays. (#8222) @AdamGS
🐛 Bug Fixes
13 changes
- fix: keep Arrow device schemas aligned with exports (#8360) @0ax1
- aggregate_fn: Return NaN instead of null for mean of all-null input (#8365) @dimitarvdimitrov
- fix[file]: read the one-row pruning result in can_prune instead of re… (#8369) @tomsanbear
- Fix generation of stat pruning expression for unsupported dtypes (#8326) @robert3005
- fix[gpu]: decode extension arrays on the GPU (#8353) @0ax1
- fix issue with FileSystem path handling (#8248) @robert3005
- Wrap object stores in python with Compat layer (#8316) @robert3005
- Fix semantic conflict in vortex-array (#8312) @AdamGS
- fix:
[profile.bench]codegen-units = 16(#8257) @joseph-isaacs - fix grouped sum index for listview array (#8255) @onursatici
- split registration to respect nested projection and filters (#8213) @onursatici
- Fix zoned min/max over fixed-size-list stats (#8243) @gatesn
- fix session deadlock (#8217) @onursatici
📖 Documentation
🧰 Maintenance
40 changes
- chore: clean up Arrow device export (#8359) @0ax1
- chore: polish cuDF C FFI (#8358) @0ax1
- Remove the unused website and clean other dependencies (#8362) @AdamGS
- Minor touchups for vortex-datafusion (#8356) @AdamGS
- Add docs for StatsRewriteRule (#8355) @AdamGS
- Disable wide-columns benchmark to stop benchmarks from timing out (#8331) @AdamGS
- Optimize
fsst_compress_offsets_overflow_i32by 2X locally (#8324) @AdamGS - Add slicing benchmarks for vortex-buffer (#8323) @AdamGS
- test[gpu]: more e2e tests (#8317) @0ax1
- Publish Parquet-Variant encoding (#8313) @AdamGS
- Fix javadoc warnings and run lints in ci (#8309) @robert3005
- Update all patch updates (#8286) @renovate[bot]
- Update Rust crate tabled to 0.21.0 (#8184) @renovate[bot]
- Update logback monorepo to v1.5.34 (#8287) @renovate[bot]
- Update taiki-e/install-action digest to 0631aa6 (#8285) @renovate[bot]
- Update actions/checkout digest to df4cb1c (#8281) @renovate[bot]
- Update anthropics/claude-code-action digest to fbda2eb (#8282) @renovate[bot]
- Lock file maintenance Rust lock file maintenance (#8296) @renovate[bot]
- Update aws-actions/configure-aws-credentials digest to e7f100c (#8283) @renovate[bot]
- Update netty monorepo to v4.2.15.Final (#8288) @renovate[bot]
- Update storybook monorepo to v10.4.2 (#8290) @renovate[bot]
- Update react monorepo (#8289) @renovate[bot]
- Update tokio-prost monorepo to v0.14.4 (#8291) @renovate[bot]
- Update typescript-eslint monorepo to v8.60.1 (#8292) @renovate[bot]
- Update tanstack-virtual monorepo to v3.14.2 (#8293) @renovate[bot]
- Lock file maintenance JS lock file maintenance (#8295) @renovate[bot]
- Update codecov/codecov-action action to v7 (#8294) @renovate[bot]
- Update codecov/codecov-action digest to fb8b358 (#8284) @renovate[bot]
- Ignore RUSTSEC-2026-0173 (#8300) @AdamGS
- Rename ArrayEq/Hash Precision to
EqMode(#8214) @AdamGS - use octet_length() instead of len() in clickbench (#8267) @myrrc
- Mask bool iterator (#8258) @onursatici
- Update dependency starlette to v1 [SECURITY] (#8254) @renovate[bot]
- bench: sweep bit-packed compare across all int types and bit widths (#8238) @joseph-isaacs
- Make
from_arrownot panic (#8242) @connortsui20 - clean: drop unused API lock-file (#8241) @0ax1
- CachedIds for functions (#8240) @myrrc
- vortex-io: Add explicit span on spawn_io (#8232) @brancz
- Add
select_unpredictablehint to search_sorted (#8225) @AdamGS - Add ArcSwapMap and use it throughout our session registries (#8072) @robert3005