v0.10.0
Immutable
release. Only release title and notes can be modified.
A vortex.zstd overhaul: compression now runs through FFM bindings to the native libzstd, gaining framed (sliceable) payloads, nullable-column support, and shared-dictionary decode. Alongside it, zone-map pruning is fixed to compare in the column's type domain.
Added
DType.isUnsigned()—truefor the unsigned integer primitives (U8–U64),falseotherwise. (#159)- The
vortex.zstdencoder now writes nullable columns (primitive and utf8/binary): null positions are stripped before compression and validity is emitted as a Bool child, matching the Rust reference layout. Whenvortex.zstdis the configured encoder, nullable primitive columns route to it directly instead of being wrapped invortex.masked. new ZstdEncodingEncoder(valuesPerFrame)splits the payload into independently compressed frames ofvaluesPerFramevalues each (oneZstdFrameMetadataper frame), letting a slice scan decompress only the frames overlapping its row range. The no-arg constructor still emits a single frame. (#170)
Changed
- The
vortex.zstdencoding now compresses and decompresses throughio.github.dfa1.zstd:zstd(FFM bindings to the nativelibzstd) instead ofio.airlift:aircompressor-v3. Consumers ofvortex.zstddeclare a single dependency,io.github.dfa1.zstd:zstd-platform, which transitively brings thezstdbinding plus the nativelibzstdfor every supported platform (replacing the former per-platformzstd-native-<platform>artifacts).
Fixed
vortex.zstdsegments compressed with a shared (trained) dictionary now decode, via the nativelibzstddictionary support, instead of being rejected. The upstreamzstd.vortexcompatibility fixture is read end-to-end and matches the Rust reference. (#104)- Writing a nullable
Utf8/Binarycolumn no longer throwsNullPointerException(or silently drops nulls): nullable string columns now carry their validity like nullable primitives and round-trip throughvortex.masked. As a result they decode asMaskedArray(validity + values child) rather than a bareVarBinArray. (#168) - CSV export now handles nullable columns (
MaskedArray): null rows export as an empty field instead of failing with "unsupported array type for CSV export". (#168) - Zone-map pruning now compares filter values in the column's type domain rather than by the boxed value's type. A predicate whose value is boxed at a different width (e.g.
Integeron anI64column) — or any value on aU64column — previously pruned nothing and silently degraded to a full scan; it now prunes correctly (unsigned columns by unsigned order). As part of this, a filter value genuinely incomparable to its column (e.g. aStringagainst a numeric column) now raisesVortexExceptionduring the scan instead of silently disabling pruning — a behaviour change for callers that relied on the previous silent full scan. (#159)