v0.7.3
Parquet ZSTD support, vortex.patched encoder, constant-encoding selection fix, Windows TUI raw-mode fix.
Added
- Parquet: ZSTD-compressed Parquet import —
zstd-jniwas an optional dep in hardwood and had to be declared explicitly. NYC Yellow Taxi 2024-01 (47.6 MB Parquet, 2.96 M rows × 19 cols) imports to 40.7 MB Vortex — 14% smaller than the Rust JNI reference (47 MB) thanks to the global-dict encoder catching low-cardinalityF64columns. - Writer:
vortex.patchedencoder — identifies outlier values that exceed the optimal bit width, zeros them in the inner array (exposed as an open cascade child for further bitpacking), and stores their within-chunk U16 indices and raw values separately.
Fixed
- CLI: Windows TUI raw-mode —
readKeynow callsReadFiledirectly on the kernel handle obtained viaGetStdHandleinstead of reading fromSystem.in. Java'sSystem.ingoes through JVM-internal CRT wrappers that ignoreSetConsoleMode, so every keypress previously required Enter before the TUI reacted. - Writer: constant encoding skipped for single-distinct-value columns —
isDictCandidatereturnedtruefordistinctCount == 1, routing all-same-value columns through the global-dict path instead ofvortex.constant.
Changed
- CLI: polling loop in
Terminal.readKey(Duration)extracted toKeyDecoder.nextWithTimeout(InputStream, Duration)— eliminates duplication betweenPosixTerminalandWindowsTerminal.
Tests
- Integration:
TaxiParquetOracleVsJavaIntegrationTest— hardwood reads the taxi Parquet to a CSV (oracle);ParquetImporter→CsvExporterproduces a second CSV (SUT); line-by-line diff must be zero. Proves the importer loses no data across 2.96 M rows × 19 columns.
Full changelog: https://github.com/dfa1/vortex-java/blob/main/CHANGELOG.md#0.7.3