Skip to content

v0.7.3

Choose a tag to compare

@dfa1 dfa1 released this 17 Jun 18:30

Parquet ZSTD support, vortex.patched encoder, constant-encoding selection fix, Windows TUI raw-mode fix.

Added

  • Parquet: ZSTD-compressed Parquet importzstd-jni was an optional dep in hardwood and had to be declared explicitly. NYC Yellow Taxi 2024-01 (47.6 MB Parquet, 2.96 M rows × 19 cols) imports to 40.7 MB Vortex — 14% smaller than the Rust JNI reference (47 MB) thanks to the global-dict encoder catching low-cardinality F64 columns.
  • Writer: vortex.patched encoder — identifies outlier values that exceed the optimal bit width, zeros them in the inner array (exposed as an open cascade child for further bitpacking), and stores their within-chunk U16 indices and raw values separately.

Fixed

  • CLI: Windows TUI raw-modereadKey now calls ReadFile directly on the kernel handle obtained via GetStdHandle instead of reading from System.in. Java's System.in goes through JVM-internal CRT wrappers that ignore SetConsoleMode, so every keypress previously required Enter before the TUI reacted.
  • Writer: constant encoding skipped for single-distinct-value columnsisDictCandidate returned true for distinctCount == 1, routing all-same-value columns through the global-dict path instead of vortex.constant.

Changed

  • CLI: polling loop in Terminal.readKey(Duration) extracted to KeyDecoder.nextWithTimeout(InputStream, Duration) — eliminates duplication between PosixTerminal and WindowsTerminal.

Tests

  • Integration: TaxiParquetOracleVsJavaIntegrationTest — hardwood reads the taxi Parquet to a CSV (oracle); ParquetImporterCsvExporter produces a second CSV (SUT); line-by-line diff must be zero. Proves the importer loses no data across 2.96 M rows × 19 columns.

Full changelog: https://github.com/dfa1/vortex-java/blob/main/CHANGELOG.md#0.7.3