Repository navigation
Releases
PARQONAUT v0.9.2 - S3 Multipart Correctness
PARQONAUT v0.9.2 — S3 multipart correctness
Fixes S3 multipart output corruption/finalization around the default 5 MiB part boundary.
When an AsyncWrite caller retried a write after Poll::Pending, the multipart writer could append the same bytes twice, producing objects with duplicated or corrupted payload (including Parquet files crossing the 5 MiB threshold).
No intended feature changes.
PARQONAUT v0.9.1 - Remote Pipeline Completeness
PARQONAUT v0.9.1 — Remote pipeline completeness
Highlights
- Bounded streaming Parquet publication to S3 — RecordBatch streams encode through a sync Parquet writer, bounded byte channels, and multipart upload finalization (no whole-object buffering).
- Complete local/S3 routing for transform and stream workflows via independent source and sink backends (
ColumnarPipelineIo). - Remote convert, split, partition, rewrite, merge, and fused specs across all four storage legs (local→local, local→S3, S3→local, S3→S3).
just pipeline-s3-demo— end-to-end RustFS pipeline: CSV schema drift → S3 convert → fused rewrite/partition → scan/verify with zero remote intermediates.- Authoritative 6×4 capability matrix integration tests on RustFS (
remote_pipeline_matrix).
Fixes
- S3 multipart
poll_shutdownnow always runsCompleteMultipartUpload/ single-part finalize instead of returning early with an empty pending queue. prqnt convertremote Parquet output useswrite_parquet_batch_streaminstead of in-memoryconditional_create.
Verification
See acceptance.md and parallel-audit.md.
PARQONAUT v0.9.0 - Unified Columnar Pipeline
PARQONAUT v0.9.0 — Unified columnar pipeline
Highlights
- Single Apache Arrow 54.3.1 / Parquet 54.3.1 stack across stream, transform, repair, and storage columnar I/O.
- Removal of arrow2 / parquet2 from product crates (
just columnar-check). - Shared
parqonaut-columnarbatch boundary:BatchSource,BatchSink, boundedBatchStream, backpressure relay. - Canonical schema compatibility in
parqonaut-columnar::schema(stream unification + repair classification). - In-memory transform spec fusion for streamable step chains with explicit barriers in dry-run plans.
- Storage-backed Parquet read/write (local and S3 range/multipart) for rewrite/merge remote paths.
- Progress event vocabulary extended on
parqonaut-workflowfor future CLI/server/TUI consumers.
Not in this release
- Plugin execution bridge → v0.10.0
- TUI and web dashboard → v0.11.0
Upgrade notes
- Stream checkpoints remain schema version 1 where compatible; stale-source resume behavior unchanged.
- Repair plan semantics (safety, authorization, manifest) are unchanged; execution machinery may reuse shared batch primitives.
PARQONAUT v0.8.1 - Transform Demo Correction
PARQONAUT v0.8.1 — Transform demo correction
Patch release correcting just transform-demo command-line flags and merge input glob so the recipe matches the released prqnt CLI.
No intended runtime behavior change from v0.8.0.
Changes
Justfile: use--output,--by, and an explicit Parquet glob for merge (was-o,--partition-by, and directory merge input)- S3 batch demos: non-overlapping fixture paths; inventory lists committed publication data for verification; batch configs accept path-style local output overrides
Upgrade
Safe to upgrade from v0.8.0 without migration steps.
PARQONAUT v0.8.0 - Transform and Streaming Workflows
PARQONAUT v0.8.0 — Transform and Streaming Workflows
Highlights
- Transform:
prqnt partition,prqnt merge,prqnt split, and declarativeprqnt transform --specmulti-step pipelines with typed plan compilation,--check, and--dry-run. - Streaming: Schema discovery, unification, and batch alignment in
prqnt convertwith strict / widen / stringify policies. - Resumability: Checkpoint schema v1 with input fingerprints, stale detection, staged output, and interruption recovery.
- Progress: Shared
ProgressEventcontract with observer implementations and--json-progresson convert. - Documentation: Transform, streaming, and pre-1.0 roadmap pages on GitHub Pages.
Arrow / Parquet convergence
Stack unification across repair, transform, and stream remains planned for v0.9.0 (docs/development/columnar-convergence-plan.md).
Upgrade from v0.7.x
- Workspace version is 0.8.0; repair plan goldens include updated
parqonaut_version. - No breaking changes to repair plan or batch journal contracts.
PARQONAUT v0.7.1 - Documentation Site
PARQONAUT v0.7.1 — Documentation Site
Documentation and delivery infrastructure release. No intended runtime behavior changes.
Highlights
- Public documentation site on GitHub Pages (mdBook 0.5.4)
- Generated CLI reference and published OpenAPI JSON + rustdoc
just docs-check/just docs-buildand dedicated documentation CI job- Engineering history remains in the repository but outside the public site navigation
Deferred
Unchanged from v0.7.0 product deferrals (distributed workers, Idempotency-Key, SDKs, web UI, TLS in serve, etc.).
PARQONAUT v0.7.0 - Application Server
PARQONAUT v0.7.0 — Application Server
Highlights
prqnt serve— local HTTP application server with/api/v1(OpenAPI, metrics, health live/ready).- Shared application layer — CLI and HTTP both call
parqonaut-appfor scan, repair, and batch workflows. - Durable jobs — scan, repair, batch repair, and batch resume with lease/recovery, cancellation, and SQLite/Postgres persistence.
- Security defaults — loopback bind, RBAC bearer tokens (
prqnt_prefix), storage location policy, bootstrap viaPRQNT_BOOTSTRAP_ADMIN_TOKEN. - S3 — API and workers use the same storage backends; RustFS-backed CI for object storage.
Deferred (not in this release)
- Distributed / remote workers
- Idempotency-Key header support
- SDK generation, web UI, TUI
- TLS termination in
prqnt serve - Plugin execution on the server
See CHANGELOG for full details.
PARQONAUT v0.6.0 - Unified prqnt CLI
PARQONAUT v0.6.0 — Unified prqnt CLI
Breaking change
Install and invoke prqnt instead of parqonaut. There is no compatibility shim.
cargo install --path crates/parqonaut-cli
prqnt --help
prqnt scan ./dataRemoved entrypoints
paraclete-httpgenerate-phase*/generate-golden-plansproduct binaries (usecargo xtask)
Developer tooling
cargo xtask fixtures repair
cargo xtask fixtures schema
cargo xtask fixtures orchestration
cargo xtask fixtures object-storage
cargo xtask golden-plansNaming cleanup
Active architecture, fixtures, scripts, Just targets, and CI jobs now use capability-oriented names. See docs/development/naming-migration-v0.6.md.
PARQONAUT v0.5.0 — Remote Object Storage
PARQONAUT v0.5.0 — Remote Object Storage
Phase 5 delivers transactional S3-compatible object storage for scan, repair, batch orchestration, and publication.
Highlights
- S3 storage backend with conditional writes, ranged reads, multipart streaming, and remote publication (
COMMITTED/CURRENT). - Mixed fleet batch orchestration — local and
s3://sources and outputs with durable SQLite journals and resume. - FOGBANK integration fixtures and chaos coverage for storage retries and publication safety.
- Bounded remote Parquet footer parsing — no allocation proportional to declared object size.
- CI:
phase5-s3-integrationjob using pinned RustFS 1.0.0 (S3-compatible), explicitdocker run, health/health/ready, AWS CLI smoke (including large-object multipart).
Upgrade notes
- Prefer
PARQONAUT_S3_ENDPOINT,PARQONAUT_S3_PATH_STYLE, andPARQONAUT_S3_BUCKET; legacyMINIO_*env vars remain as aliases for local scripts. - Production remote repair uses
S3StorageBackendviaBatchStorageRuntime::new(); in-memory remote backends are test-only.
PARQONAUT v0.4.1 — Release Metadata Correction
Release metadata correction only — no functional changes from v0.4.0.
Fixed
- Workspace crate version aligned with published release tags (
0.3.0→0.4.1) CHANGELOG.mdupdated with missing entries for v0.2.0–v0.4.0README.mdupdated for Phase 4 batch orchestration capabilities
Verification
just cijust phase3-demojust phase4-demojust phase4-resume-demo