Skip to content

v0.10.0 — write-side polish + opportunistic deferred items

Choose a tag to compare

@ryan-evans-git ryan-evans-git released this 20 May 00:53
· 56 commits to main since this release
30dfa31

v0.10.0 bundles every item deferred under v0.9.x "Scope notes" plus the below-the-phase-line catch-all list, plus an I/O unblock that lets parallel decode scale honestly.

Write-side completeness

  • Multi-column / multi-RG bloom writes (#54) — per-(RG, col) SBBFs in write_table_with_blooms_to_path
  • Bloom on PLAIN (non-dict) write paths (#56) — write_{i32,i64,f64,byte_array}_column_with_bloom_to_path family
  • Multi-column dict writes (#57) — per-column opt-in via dict_per_column: &[bool]
  • Per-column codec + WriteOptions (#59) — new write_table_with_options_to_path bundling row_group_size, page_version, default_codec, codec_per_column, dict_per_column, bloom_fpps. Different columns in the same row group can use different codecs

Decode coverage

  • DELTA_BINARY_PACKED u64-output unpacker (#58) — unpack_indices64_into covers bit_widths 1..=64; decode_delta_i64 no longer errors on streams with bit_width > 32
  • BYTE_ARRAY batched/streaming decode API (#60) — read_column_byte_array_batches mirrors the scalar batched API for Vec<Vec<u8>>

Perf

  • pread-based unlocked I/O (#55) — no more Mutex<File> on parallel workers; ~28% sequential decode speedup as a side effect
  • NEON pld L1 prefetch hints in dict gather (#61) — behaviour-identical, perf insensitive to dict footprint up to 1 MB
  • NEON unpackers for bw=4 + bw=8 (#62)

Full test suite (663 tests) green at the new version.