qvd2parquet v0.5.0
Added
- Apache License 2.0. The project shipped without a licence file, which left
its terms unstated. --timezone=nonewrites timestamps with no timezone (Parquet
isAdjustedToUTC=false), preserving the QVD's naive wall clock so that every
reader shows the same value regardless of where the file is converted or
read.naiveis accepted as a synonym.- A zoned conversion now stamps
tz=UTCrather than the zone it was given.
--timezonestates which zone the input wall clocks were recorded in, not how
to label the output, and what gets stored once they are on the timeline is a
UTC instant. Stamping the source zone made Arrow readers render the values
back in it while engines reading the Parquet type alone -- which carries no
name -- rendered the instant, so identical bytes showed two different times.
--timezone=noneis unaffected and still writes no zone. - A zoned conversion now reports where it had to alter a wall clock. Twice a
year a DST change skips an hour, so a reading in it does not exist and gets
moved, or repeats an hour, so a reading in it has two instants and one is
chosen. A QVD names no timezone, which makes both changes a consequence of
the--timezoneclaim rather than of the data, so they are no longer silent.
Removed
IMPLEMENTATION_PLAN.md. It described the build that has since happened and
had drifted from the code; the README and CHANGELOG carry what is still true.
It remains in the git history.
Changed
-
Breaking.
--timezonenow defaults tonone, so timestamps are written
as the naive wall clock the QVD actually holds and nothing is converted
unless asked. The previous default,Local, interpreted every reading in the
converting machine's timezone, which made the output depend on where it ran:
the same QVD produced three different instants on boxes in UTC, New York and
Tokyo, none of them correct unless the data happened to come from that zone.
Pass--timezone=Localto restore the old behaviour, or name the zone the
data was recorded in to get true instants. -
Options.TimezoneNameis now authoritative inValidate, which derives
LocationandNaiveTimestampsfrom it. Setting one of those fields alone no
longer leaves the conversion disagreeing with the type. -
Timestamps are now
timestamp[us]rather thantimestamp[ms]. A Qlik serial
resolves to about 0.63us at present-day dates, so milliseconds discarded real
signal while still carrying the encoding noise that makes a stored07:15:00
surface as07:14:59.999999. Rounding to the microsecond keeps the precision
and removes the noise. -
Releases now ship Linux, Windows and macOS on
amd64andarm64only. The
32-bit,arm,ppc64le,s390x,riscv64and BSD targets were building
and shipping without being asked for. Another target is a matter of listing
it inscripts/build-release.shand the two workflow matrices, which a test
keeps identical. -
The release workflow builds its targets in parallel, one job each, instead of
looping through them in a single job. Together with the trimmed list that
takes a release from about twenty minutes to a couple.
Install
Download the archive for your platform below, unpack it, and put qvd2parquet
on your PATH. Verify the download against SHA256SUMS:
shasum -a 256 -c SHA256SUMS --ignore-missingBinaries are pure Go and statically linked, so they need no runtime
dependencies: Linux (amd64, arm64), Windows (amd64, arm64) and macOS (amd64,
arm64).