Skip to content

v0.18.51

Choose a tag to compare

@derekmwright derekmwright released this 06 Sep 14:32

v0.18.51

Parquet hardening — a wave of reader/writer corruption gaps, each found by an
adversarial mutation of a file Wadjet itself wrote and read back through the
public reader, and each now refused instead of returning wrong data.

  • Parquet writer: a short write from the underlying io.Writer (fewer bytes
    taken than handed over, with no error) is detected and reported as a failure
    instead of silently finalizing a truncated file; the failure latches, so later
    writes and Close stay failed. (#926)
  • Parquet reader: a v1 data page whose level section decodes fewer values than
    it declares is refused, rather than reinterpreting the level bytes as column
    values. (#923)
  • Parquet reader: a dictionary page encountered in the data-page walk is refused,
    rather than being skipped after its rows were already charged (which dropped
    the tail of a column). (#924)
  • Parquet reader: row-group statistics are keyed by full leaf path, so a nested
    field no longer overwrites a same-named top-level column's bounds — which could
    wrongly prune a row group. (#925)
  • Parquet reader: each row-group column chunk is bound to its schema leaf by full
    path; footer metadata whose columns are swapped, duplicated, missing, unknown,
    or contradict the schema is refused, rather than silently returning one
    column's values under another's name. (#927)
  • Scan: predicate pruning resolves a column by its full path, so a query on a
    top-level column is never pruned away by a nested field that shares its name.
    (#915)