Skip to content

qvd2parquet v0.4.0

Choose a tag to compare

@github-actions github-actions released this 21 Aug 22:39
· 35 commits to main since this release
6b1863a

Added

  • --out-dir converts a whole folder: pass files or directories, and each
    .qvd becomes a .parquet of the same name. A failing file does not stop
    the run; every input is attempted, failures are listed at the end, and the
    exit code reports the most actionable one. --recursive descends into
    subdirectories. Two inputs that would produce the same output file are
    refused before anything is written, since --force would otherwise silently
    overwrite the first result.

  • --file-workers converts several files at once and divides the decode
    workers between them, so the total stays near one per CPU. This is why folder
    conversion is built in rather than left to a shell loop: separate processes
    would each start NumCPU workers and oversubscribe the machine.

  • --log writes JSON Lines, one record per file plus a summary, so a finished
    run can be queried with DuckDB or jq rather than read. Each record carries
    row and column counts, output size, elapsed time, throughput, and the quality
    gate's verdict. In batch mode --schema-report and --quality-report write
    one document per input, named after it.

  • --inspect reads the XML header and the symbol tables, prints the schema a
    conversion would produce, and exits without touching the record area. The
    cost is independent of row count: on a 29 MiB, 5-million-row file it reads
    7.9 KiB and finishes in 0.01s, against 1.58s for the full conversion. All
    type policy flags apply, so the report shows what a conversion would write.
    A file the type policy rejects prints the reason plus the raw symbol profiles
    that explain it and exits 3, which makes it a cheap pre-flight check. The
    report goes to stdout; --schema-report also works in inspect mode.

  • Qlik's semantic field tags are now read. A field declaring no
    NumberFormat/Type but tagged $date, $timestamp, $time or $interval
    is resolved to that type. This works for plain numeric fields carrying no
    dual display strings, which no amount of text inspection could identify, and
    it is what Qlik Sense writes. A declared type still wins over a tag.

  • An empty string symbol is written as null, which is how Qlik treats it.
    The substitution is counted and reported. Pass --empty-as-null=false to
    keep "" distinct from null.

  • A NaN or infinite value in a date, timestamp, time, integer or decimal
    column is written as null rather than failing the conversion. Such a
    value is not something those types can hold, and nothing is lost by nulling
    it. The substitution is counted and reported on stderr and in
    --schema-report, so it is never silent. A finite value that simply does
    not fit is still an error, because nulling it would discard real data.
    float64 columns keep NaN and infinity, which they can represent.

  • A date or timestamp column is now validated when the schema is resolved,
    whatever decided its type -- the declared header, a Qlik tag, or display-string
    inference. A value that cannot be converted fails as a schema policy error
    naming the column and the value, so --inspect predicts it and no output file
    is started, instead of the conversion failing part-way through.

  • --infer-dates (on by default) is the fallback for files that carry no tags:
    a column with no declared type is read as a date or timestamp when every
    display string renders its Excel-style serial value as one. The check is
    format-agnostic and accepts a neighbouring day, since the string was rendered
    in whatever timezone wrote the file and a whole-hour offset can move the
    calendar date. Serials outside roughly 1900 to 2200 are never read as dates,
    blank display strings are not evidence, and a column mixing dates with
    anything else is left alone.

    Only words that belong to a rendered date -- month and weekday names in
    English and German, meridiem markers, ordinal suffixes and timezone
    abbreviations -- may appear alongside the digits, and a month or weekday name
    that contradicts the value is rejected, so "Mon, 20 Nov 2010" beside a
    Saturday keeps its text. The ISO 8601 T between date and time is treated as
    punctuation. A clock time uses a narrower list still, since a month or weekday
    name is not something a time32 value can encode. So "Due 11/20/2010" and "20 Jan 2010"
    beside a November value both keep their text column. The list errs short: a
    word wrongly rejected costs a redundant column, whereas one wrongly accepted
    drops text that carried information.

Changed

  • --dual now defaults to auto. A Qlik dual's display string is written
    as a ${name}__text column only when it carries something the numeric column
    does not. A localized number such as 1.234,56, or a date rendered beside a
    column that already encodes it, is redundant and dropped; a label such as
    Open beside 1 is kept, and the reason is reported. One informative string
    is enough to keep the column, so the default errs towards preserving data.
    --dual=numeric, text and columns still force a choice.

    Redundancy is judged against the value that will actually be written, not the
    raw payload, so a MONEY field carrying 1.234 and displaying 1.23 at
    scale 2 does not produce a text column.

  • The banner now carries the year: (c) 2026, RALFORION d.o.o.

Install

Download the archive for your platform below, unpack it, and put qvd2parquet
on your PATH. Verify the download against SHA256SUMS:

shasum -a 256 -c SHA256SUMS --ignore-missing

Binaries are pure Go and statically linked, so they need no runtime
dependencies: Linux (amd64, arm64), Windows (amd64, arm64) and macOS (amd64,
arm64).