Skip to content

Releases

v0.3.1 — dependency security fixes

Choose a tag to compare

@github-actions github-actions released this 01 Nov 09:29

Dependency fixes only; no behaviour changes. Upgrading from v0.3.0 is a binary swap.

What changed

  • A lockfile you can build from. The lockfile had drifted from the manifests since August, so cargo --locked refused it, and v0.3.0's binaries were built with dependencies resolved at build time. v0.3.1 is the first release built from a committed, checked lockfile.
  • rustls 0.23.46 fixes four TLS advisories:
    • TLS 1.3 handshake messages accepted across encryption levels;
    • secrets in a session's Debug output;
    • unbounded memory in brotli certificate decompression;
    • a ServerHello that poisons the per-name key-exchange record.
  • arrow and parquet 58.4.1, which moves thrift from 0.17 to 0.23. quinn-proto 0.11.16. Together they clear both open Dependabot alerts.
  • docs/arrow-versions.md: parket uses Arrow 58 throughout. The doc explains why moving to deltalake 1.x, which is on Arrow 59, waits on connector_arrow.

Downloads

  • Static binaries (musl) and glibc builds for x86_64 and aarch64 are attached below, with checksums-sha256.txt.
  • Container image: ghcr.io/oonid/parket:0.3.1, which is also :latest.

Pull requests

  • #7 Bring Cargo.lock in sync and update rustls to 0.23.46
  • #8 Fix the quinn-proto and thrift advisories; record the Arrow version split
  • #9 Release v0.3.1

Full changelog: v0.3.0...v0.3.1

v0.3.0 — archive mode

Choose a tag to compare

@github-actions github-actions released this 30 Oct 07:36

Archive mode, and safety checks that change how some runs end. A minor version: archive mode is new, and a two-stream sync that used to commit an inconsistent rewrite now stops before committing.

Highlights

  • Archive mode. A table set to TABLE_ARCHIVE_<t>=<scope> is never synced. It is copied to Delta one unit at a time, so the Delta copy can outlive the source rows. Three commands work on units:

    • --archive-units appends each unit in one commit, stamped with _archive_unit. Timestamps are stored as UTC, with the source's zone as metadata.
    • --verify-archive-units compares a unit with the source row by row.
    • --replace-archive-units repairs a unit by replacing exactly its files.

    A library API serves callers that choose the units themselves. A scope may use the unit id more than once (id >= ? * 10000 AND id < (? + 1) * 10000), so a table without a unit column can be archived in ranges of its primary key. See config.md and cli.md.

  • Two-stream upsert overwrites check their row accounting before committing. Kept plus replaced rows must equal the rows removed, and the staged rows must equal kept plus new rows. Otherwise the run stops and the live table is untouched.

  • A Drift verdict compares values per key bucket and names the buckets that differ. It reports only.

  • Every remedy that offers a full refresh or TABLE_RECONCILE says what it costs: the Delta table is replaced by the source's current rows, dropping rows the source has deleted or purged.

  • --vacuum tells files the log names apart from files present in storage. Its byte totals were already right.

Upgrading from v0.2.6

  1. Run parket --check with the new binary.
  2. Run one supervised sync.
  3. Keep v0.2.6 beside it for a one-step rollback.

A sync that now stops on the row-accounting check has left the table as it was. Its message says what did not add up.

Downloads

  • Static binaries (musl) and glibc builds for x86_64 and aarch64 are attached below, with checksums-sha256.txt.
  • Container image: ghcr.io/oonid/parket:0.3.0, which is also :latest.

Pull requests

  • #1 Pre-archive safety: overwrite warnings, upsert row accounting, bucketed Drift value check
  • #2 Describe tooling and review in neutral terms
  • #3 Archive mode: append, verify and repair units of archive-only tables
  • #4 Use checked_div for the per-row memory estimate
  • #5 Let a scope use the unit id more than once
  • #6 Release v0.3.0

Full changelog: v0.2.6...v0.3.0

v0.2.6

Choose a tag to compare

@github-actions github-actions released this 07 Aug 11:58

Full Changelog: v0.2.5...v0.2.6

v0.2.5

Choose a tag to compare

@github-actions github-actions released this 07 Aug 08:15

Full Changelog: v0.2.4...v0.2.5

v0.2.4

v0.2.4 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 07 Aug 04:36

Full Changelog: v0.2.3...v0.2.4

v0.2.3

Choose a tag to compare

@github-actions github-actions released this 05 Aug 05:51

Full Changelog: v0.2.2...v0.2.3

v0.2.2

Choose a tag to compare

@github-actions github-actions released this 18 Jul 08:33

Full Changelog: v0.2.1...v0.2.2

v0.2.1

Choose a tag to compare

@github-actions github-actions released this 16 Jul 06:09

Full Changelog: v0.2.0...v0.2.1

v0.2.0

Choose a tag to compare

@github-actions github-actions released this 15 Jul 07:56

Full Changelog: v0.1.2...v0.2.0

v0.1.2

Choose a tag to compare

@github-actions github-actions released this 10 Jul 12:08

Full Changelog: v0.1.1...v0.1.2