Skip to content

Releases

PARQONAUT v0.9.2 - S3 Multipart Correctness

Choose a tag to compare

@sempervent sempervent released this 21 Sep 23:59
a786901

PARQONAUT v0.9.2 — S3 multipart correctness

Fixes S3 multipart output corruption/finalization around the default 5 MiB part boundary.

When an AsyncWrite caller retried a write after Poll::Pending, the multipart writer could append the same bytes twice, producing objects with duplicated or corrupted payload (including Parquet files crossing the 5 MiB threshold).

No intended feature changes.

PARQONAUT v0.9.1 - Remote Pipeline Completeness

Choose a tag to compare

@sempervent sempervent released this 21 Sep 00:55
a786901

PARQONAUT v0.9.1 — Remote pipeline completeness

Highlights

  • Bounded streaming Parquet publication to S3 — RecordBatch streams encode through a sync Parquet writer, bounded byte channels, and multipart upload finalization (no whole-object buffering).
  • Complete local/S3 routing for transform and stream workflows via independent source and sink backends (ColumnarPipelineIo).
  • Remote convert, split, partition, rewrite, merge, and fused specs across all four storage legs (local→local, local→S3, S3→local, S3→S3).
  • just pipeline-s3-demo — end-to-end RustFS pipeline: CSV schema drift → S3 convert → fused rewrite/partition → scan/verify with zero remote intermediates.
  • Authoritative 6×4 capability matrix integration tests on RustFS (remote_pipeline_matrix).

Fixes

  • S3 multipart poll_shutdown now always runs CompleteMultipartUpload / single-part finalize instead of returning early with an empty pending queue.
  • prqnt convert remote Parquet output uses write_parquet_batch_stream instead of in-memory conditional_create.

Verification

See acceptance.md and parallel-audit.md.

PARQONAUT v0.9.0 - Unified Columnar Pipeline

Choose a tag to compare

@sempervent sempervent released this 20 Sep 19:45
7d40bfe

PARQONAUT v0.9.0 — Unified columnar pipeline

Highlights

  • Single Apache Arrow 54.3.1 / Parquet 54.3.1 stack across stream, transform, repair, and storage columnar I/O.
  • Removal of arrow2 / parquet2 from product crates (just columnar-check).
  • Shared parqonaut-columnar batch boundary: BatchSource, BatchSink, bounded BatchStream, backpressure relay.
  • Canonical schema compatibility in parqonaut-columnar::schema (stream unification + repair classification).
  • In-memory transform spec fusion for streamable step chains with explicit barriers in dry-run plans.
  • Storage-backed Parquet read/write (local and S3 range/multipart) for rewrite/merge remote paths.
  • Progress event vocabulary extended on parqonaut-workflow for future CLI/server/TUI consumers.

Not in this release

  • Plugin execution bridge → v0.10.0
  • TUI and web dashboard → v0.11.0

Upgrade notes

  • Stream checkpoints remain schema version 1 where compatible; stale-source resume behavior unchanged.
  • Repair plan semantics (safety, authorization, manifest) are unchanged; execution machinery may reuse shared batch primitives.

PARQONAUT v0.8.1 - Transform Demo Correction

Choose a tag to compare

@sempervent sempervent released this 20 Sep 04:45
9904898

PARQONAUT v0.8.1 — Transform demo correction

Patch release correcting just transform-demo command-line flags and merge input glob so the recipe matches the released prqnt CLI.

No intended runtime behavior change from v0.8.0.

Changes

  • Justfile: use --output, --by, and an explicit Parquet glob for merge (was -o, --partition-by, and directory merge input)
  • S3 batch demos: non-overlapping fixture paths; inventory lists committed publication data for verification; batch configs accept path-style local output overrides

Upgrade

Safe to upgrade from v0.8.0 without migration steps.

PARQONAUT v0.8.0 - Transform and Streaming Workflows

Choose a tag to compare

@sempervent sempervent released this 20 Sep 01:48
d3fdf1d

PARQONAUT v0.8.0 — Transform and Streaming Workflows

Highlights

  • Transform: prqnt partition, prqnt merge, prqnt split, and declarative prqnt transform --spec multi-step pipelines with typed plan compilation, --check, and --dry-run.
  • Streaming: Schema discovery, unification, and batch alignment in prqnt convert with strict / widen / stringify policies.
  • Resumability: Checkpoint schema v1 with input fingerprints, stale detection, staged output, and interruption recovery.
  • Progress: Shared ProgressEvent contract with observer implementations and --json-progress on convert.
  • Documentation: Transform, streaming, and pre-1.0 roadmap pages on GitHub Pages.

Arrow / Parquet convergence

Stack unification across repair, transform, and stream remains planned for v0.9.0 (docs/development/columnar-convergence-plan.md).

Upgrade from v0.7.x

  • Workspace version is 0.8.0; repair plan goldens include updated parqonaut_version.
  • No breaking changes to repair plan or batch journal contracts.

PARQONAUT v0.7.1 - Documentation Site

Choose a tag to compare

@sempervent sempervent released this 19 Sep 22:01

PARQONAUT v0.7.1 — Documentation Site

Documentation and delivery infrastructure release. No intended runtime behavior changes.

Highlights

  • Public documentation site on GitHub Pages (mdBook 0.5.4)
  • Generated CLI reference and published OpenAPI JSON + rustdoc
  • just docs-check / just docs-build and dedicated documentation CI job
  • Engineering history remains in the repository but outside the public site navigation

Deferred

Unchanged from v0.7.0 product deferrals (distributed workers, Idempotency-Key, SDKs, web UI, TLS in serve, etc.).

PARQONAUT v0.7.0 - Application Server

Choose a tag to compare

@sempervent sempervent released this 19 Sep 20:52
7b7f958

PARQONAUT v0.7.0 — Application Server

Highlights

  • prqnt serve — local HTTP application server with /api/v1 (OpenAPI, metrics, health live/ready).
  • Shared application layer — CLI and HTTP both call parqonaut-app for scan, repair, and batch workflows.
  • Durable jobs — scan, repair, batch repair, and batch resume with lease/recovery, cancellation, and SQLite/Postgres persistence.
  • Security defaults — loopback bind, RBAC bearer tokens (prqnt_ prefix), storage location policy, bootstrap via PRQNT_BOOTSTRAP_ADMIN_TOKEN.
  • S3 — API and workers use the same storage backends; RustFS-backed CI for object storage.

Deferred (not in this release)

  • Distributed / remote workers
  • Idempotency-Key header support
  • SDK generation, web UI, TUI
  • TLS termination in prqnt serve
  • Plugin execution on the server

See CHANGELOG for full details.

PARQONAUT v0.6.0 - Unified prqnt CLI

Choose a tag to compare

@sempervent sempervent released this 19 Sep 16:51
7f8170d

PARQONAUT v0.6.0 — Unified prqnt CLI

Breaking change

Install and invoke prqnt instead of parqonaut. There is no compatibility shim.

cargo install --path crates/parqonaut-cli
prqnt --help
prqnt scan ./data

Removed entrypoints

  • paraclete-http
  • generate-phase* / generate-golden-plans product binaries (use cargo xtask)

Developer tooling

cargo xtask fixtures repair
cargo xtask fixtures schema
cargo xtask fixtures orchestration
cargo xtask fixtures object-storage
cargo xtask golden-plans

Naming cleanup

Active architecture, fixtures, scripts, Just targets, and CI jobs now use capability-oriented names. See docs/development/naming-migration-v0.6.md.

PARQONAUT v0.5.0 — Remote Object Storage

Choose a tag to compare

@sempervent sempervent released this 19 Sep 15:36

PARQONAUT v0.5.0 — Remote Object Storage

Phase 5 delivers transactional S3-compatible object storage for scan, repair, batch orchestration, and publication.

Highlights

  • S3 storage backend with conditional writes, ranged reads, multipart streaming, and remote publication (COMMITTED / CURRENT).
  • Mixed fleet batch orchestration — local and s3:// sources and outputs with durable SQLite journals and resume.
  • FOGBANK integration fixtures and chaos coverage for storage retries and publication safety.
  • Bounded remote Parquet footer parsing — no allocation proportional to declared object size.
  • CI: phase5-s3-integration job using pinned RustFS 1.0.0 (S3-compatible), explicit docker run, health /health/ready, AWS CLI smoke (including large-object multipart).

Upgrade notes

  • Prefer PARQONAUT_S3_ENDPOINT, PARQONAUT_S3_PATH_STYLE, and PARQONAUT_S3_BUCKET; legacy MINIO_* env vars remain as aliases for local scripts.
  • Production remote repair uses S3StorageBackend via BatchStorageRuntime::new(); in-memory remote backends are test-only.

PARQONAUT v0.4.1 — Release Metadata Correction

Choose a tag to compare

@sempervent sempervent released this 19 Sep 00:02
480b031

Release metadata correction only — no functional changes from v0.4.0.

Fixed

  • Workspace crate version aligned with published release tags (0.3.0 → 0.4.1)
  • CHANGELOG.md updated with missing entries for v0.2.0–v0.4.0
  • README.md updated for Phase 4 batch orchestration capabilities

Verification

  • just ci
  • just phase3-demo
  • just phase4-demo
  • just phase4-resume-demo