Skip to content

qvd2parquet v0.3.0

Choose a tag to compare

@github-actions github-actions released this 21 Aug 17:55
· 48 commits to main since this release

Changed

  • --numeric-promote now defaults to decimal. A column carrying
    fractional values resolves to an exact Parquet decimal rather than float64.
    This is the right type for the price columns QlikView declares as plain
    REAL, where the header carries no usable scale: nDec holds a filler value
    (commonly 14) and the display format has no decimal separator. The scale is
    derived from the values themselves — the smallest at which every value is
    exactly representable, bounded at 9 decimal places.

    Pure-integer columns stay int64, since decimal(p,0) gains nothing, and a
    declared MONEY/FIX keeps using its own nDec.

    When no scale within the bound represents every value, the column really is
    floating point and is written as float64. That fallback is silent for the
    default, because a default is a preference rather than a demand; passing
    --numeric-promote=decimal explicitly makes it a demand that fails instead.

    Restore the previous behaviour with --numeric-promote=true.

    An inferred scale describes the data actually present, so a later extract
    containing more decimals can resolve the same column to a different scale.
    Pin the column with --schema where a stable schema matters.

  • --decimal-strict now defaults to false. A value that does not fit its
    declared scale is rounded to it, half away from zero — the same value Qlik
    displays for a field with nDec decimals. Rounding is counted and reported on
    stderr and in --schema-report, so it is never silent. A value is still never
    dropped.

    Restore the previous behaviour with --decimal-strict, which --strict also
    implies.

Added

  • --exclude takes comma-separated shell-style wildcard patterns and skips
    matching fields, e.g. --exclude '%*' to drop QlikView's internal key fields
    from a SAP extract. Patterns match the original QVD name, before renaming.
    Matching has no path semantics, since QVD field names routinely contain /
    and \. Excluding every column is an error.
  • --field-regex rewrites composite field names such as
    A057-||-DATBI-||-Ende Gültigkeit. Capture groups named name and comment
    are used by default, so no other flag is needed; --field-name and
    --field-comment accept Go regexp templates when named groups are not enough.
    A field the expression does not match keeps its name. The description is
    written as Parquet field metadata and the original QVD name is preserved under
    qvd.field, so nothing is lost. Two fields collapsing to one name are
    rejected as a schema policy error, and a regex that would blank every name is
    rejected up front.

Install

Download the archive for your platform below, unpack it, and put qvd2parquet
on your PATH. Verify the download against SHA256SUMS:

shasum -a 256 -c SHA256SUMS --ignore-missing

Binaries are pure Go and statically linked, so they need no runtime
dependencies: Linux (amd64, arm64, 386, arm, ppc64le, s390x, riscv64),
Windows (amd64, arm64, 386), macOS (amd64, arm64), FreeBSD (amd64, arm64),
NetBSD and OpenBSD (amd64).

Full Changelog: https://github.com/ralforion/qvd2parquet/commits/v0.3.0