Skip to content

Releases: trickle-labs/pg-aqueduct

v0.20.0

Choose a tag to compare

@github-actions github-actions released this 20 May 20:23
a538ad9

This release delivers the three high-priority architectural improvements identified in the Phase 4 engineering assessment: typed executor configuration, full multi-tenant catalog parameterisation, and automated crash recovery via the compensating-step registry.

The ExecutionContext struct replaces the optional builder pattern for the two safety-critical executor fields (desired_state and connection_string), making it a compile error to omit them. All command handlers (apply, rollback, promote) have been updated accordingly.

Every catalog SQL constant now accepts a CatalogSchema parameter and substitutes the schema name via for_schema(). The validate_catalog_schema_not_overridden() stop-gap from v0.19 is removed. aqueduct init --schema my_catalog creates all catalog tables in my_catalog; the multi-tenant isolation integration test verifies that two projects on the same database with different catalog schemas have fully independent version histories.

The --resume flow now queries aqueduct.ddl_log for rows in status = 'running' before advancing to the checkpoint. For each such row the compensating SQL is executed and the row is updated to status = 'compensated'. A new integration test covers the crash-after-DDL, before-RecordSnapshot scenario.

The OutputEmitter is now stored as a process-wide singleton via OnceLock, initialised in main before command dispatch and accessible via output::emitter(). The init command is the first to use the emitter for its success messages.

The httpmock dependency was replaced with wiremock in v0.17, eliminating the last cargo audit --deny warnings finding (RUSTSEC-2025-0052).

The catalog schema version advances from 8 to 9, adding a status column to aqueduct.ddl_log used by compensating-step recovery.

Added

  • Typed ExecutionContext struct (ARCH-2): ExecutionContext struct with required
    desired_state and connection_string fields replaces the optional builder pattern.
    Omitting either field is now a compile error. for_apply(), for_rollback(), and
    for_promote() constructors updated.
  • Full catalog schema parameterisation (ARCH-1): CatalogSchema threaded through
    every catalog SQL function. ensure_catalog_current(), connect_and_migrate(),
    read_live_state(), get_latest_dag_version(), detect_extension_installed(),
    read_ddl_log(), import_from_live(), compute_promotion_plan(),
    validate_source_clean(), and destroy_project() all accept &CatalogSchema.
    validate_catalog_schema_not_overridden() stop-gap removed.
  • Multi-tenant catalog isolation test: test_multi_tenant_catalog_isolation verifies
    that two catalogs (tenant_a, tenant_b) on the same database have independent
    dag_versions tables with no cross-contamination.
  • Automated compensating-step recovery (CORR-2): find_resume_step() queries
    aqueduct.ddl_log for status = 'running' rows and executes their compensating_sql
    via batch_execute before advancing to the resume checkpoint. Rows are updated to
    status = 'compensated' on success.
  • COMPLETE_COMPENSATING_STEP_SQL and MARK_COMPENSATING_APPLIED_SQL constants in
    catalog.rs for the two-phase compensating-step lifecycle.
  • CORR-2 crash recovery integration test: test_corr2_compensating_step_recovery
    inserts a synthetic status = 'running' ddl_log row and verifies that --resume moves
    it out of the running state.
  • Catalog schema version 9: CATALOG_SCHEMA_VERSION advances from 8 to 9. The v8→v9
    migration adds a status TEXT NOT NULL DEFAULT 'complete' column to aqueduct.ddl_log
    and backfills existing rows.
  • OutputEmitter process-wide singleton (M-1): output::init_emitter(mode) stores
    the emitter in a static OnceLock<OutputEmitter>; output::emitter() returns the
    global reference. main.rs calls init_emitter before dispatch. The init command
    uses the emitter for its success messages, replacing direct println! calls.

Changed

  • for_schema(sql, schema) now substitutes = 'aqueduct' string-literal comparisons
    and CREATE/DROP SCHEMA IF NOT EXISTS aqueduct statements in addition to the
    aqueduct. dot-prefix references.
  • DestroyOptions gains a catalog_schema: CatalogSchema field.
  • All cargo check --tests call sites for read_live_state, get_latest_dag_version,
    detect_extension_installed, read_ddl_log, import_from_live,
    compute_promotion_plan, validate_source_clean, and DestroyOptions::new updated
    throughout CLI commands and integration tests.
  • Workspace version bumped 0.19.00.20.0.

Fixed

  • executor.rs:import_from_live was calling read_live_state(client, None) (2-arg form)
    instead of the 3-arg read_live_state(client, None, catalog_schema). Fixed.
  • Extra trailing semicolons (await?;;) in commands/promote.rs and commands/status.rs
    introduced by the multi-replace pass. Removed.

v0.19.0

Choose a tag to compare

@github-actions github-actions released this 20 May 18:46
5ee835b

This release eliminates every structural correctness bug identified in the Phase 4 engineering assessment, makes the blue/green view swap fully atomic, and ensures every known correctness regression path is covered by a test that would catch it.

The headline change is that the deployment status row update in SwapConsumerViews is now executed inside the surrounding BEGIN/COMMIT block, eliminating the window where consumer views point at the green schema while the deployment row still shows status = 'active'. If the transaction is rolled back for any reason, both the view assignments and the deployment row revert together.

The heartbeat task is now supervised by a JoinHandle-based wrapper. If the heartbeat task panics — something that could previously produce a silent hang — the executor immediately receives a LockLost signal, stops execution, and surfaces the failure. This closes the last known path to a silent runaway executor.

The mock pg_trickle DDL gains a pgtrickle.paused_nodes table so integration tests can assert precisely which nodes were paused during DDL and that the scheduler was fully resumed after apply or after a failure. Two new test-kit helpers (assert_scheduler_idle and assert_scheduler_paused_for) make these assertions one-liners.

Four executor and CLI quality fixes round out the release: the WaitForConvergence poll query is now guarded by a statement_timeout to prevent a hung pg_trickle query from holding the loop past the configured deadline; status --watch applies exponential backoff on repeated connection failures; the EXPLAIN trade-off in cost.rs is documented with a code comment; and aqueduct init and aqueduct apply now return an explicit NotYetImplemented error when catalog_schema is set to anything other than "aqueduct", preventing silent data mixing in multi-tenant deployments until full schema parameterisation ships in v0.20.

All ten failure-mode test cases identified in the Phase 4 assessment are now present and passing.

Added

  • Blue/green atomicity (ARCH-3): SWAP_BLUE_GREEN_SQL (the blue_green_deployments
    status row update) is now executed inside the SwapConsumerViews transaction. A failed
    mid-swap now rolls back both view assignments and the deployment status atomically.
    Integration test swap_consumer_views_atomicity verifies this.
  • Heartbeat panic supervisor (CORR-3): tokio::spawn(run_heartbeat(...)) is wrapped in
    a JoinHandle-based supervisor that calls lock_lost_tx.send(true) if the heartbeat
    task panics. Integration test heartbeat_panic_signals_lock_loss verifies the signal.
  • Observable scheduler mock (TEST-3): mock_pgtrickle.rs gains a
    pgtrickle.paused_nodes(node_name text PRIMARY KEY, paused_at timestamptz) table.
    pause_scheduler inserts into it; resume_scheduler deletes from it.
  • Test-kit helpers: assert_scheduler_idle(client) and
    assert_scheduler_paused_for(client, nodes) added to aqueduct-testkit. Integration
    tests mock_scheduler_records_pause and mock_scheduler_idle_after_apply demonstrate
    usage.
  • statement_timeout guard for WaitForConvergence (M-9): the poll SELECT is
    wrapped in BEGIN READ ONLY; SET LOCAL statement_timeout = '{timeout}ms' so a hung
    pg_trickle query cannot hold the loop open past the configured deadline. Integration
    test waitforconvergence_deadline_respected verifies the deadline is enforced.
  • Exponential backoff in status --watch (M-8): consecutive connection or poll
    failures back off up to 5 × interval before retrying; the counter resets on the next
    successful poll.
  • EXPLAIN trade-off comment (M-4): cost.rs::estimate_rows now documents why
    format!("EXPLAIN...") is intentional and safe. The EXPLAIN call is also wrapped in a
    BEGIN READ ONLY; SET LOCAL statement_timeout = '5s' block.
  • Catalog schema stop-gap (ARCH-1 partial): validate_catalog_schema_not_overridden
    in catalog.rs returns AqueductError::NotYetImplemented { feature: "catalog_schema override" }
    for any value other than "aqueduct". Called from aqueduct init and aqueduct apply
    to prevent silent data mixing until full multi-tenant support lands in v0.20.
    Integration test catalog_schema_override_rejected_until_implemented verifies this.
  • Failure-mode tests (Phase 4): all ten test cases identified in the assessment are
    now present and passing:
    consumer_sql_injection_rejected_at_apply,
    heartbeat_panic_signals_lock_loss,
    swap_consumer_views_atomicity,
    plan_fail_on_drift_counts_consumer_deltas,
    destroy_auto_migrates_catalog,
    catalog_schema_override_rejected_until_implemented,
    age_identity_file_traversal_rejected,
    cnpg_preview_requires_valid_cert,
    waitforconvergence_deadline_respected,
    mock_scheduler_records_pause.
  • AqueductError::NotYetImplemented { feature } variant added (error code 1308).
  • cnpg_preview_requires_valid_cert unit test in preview.rs verifies that a
    create_preview_cnpg call to a plain-HTTP endpoint fails with a TLS/connection error
    when CNPG_INSECURE_SKIP_VERIFY is absent (SEC-2).

Changed

  • Workspace version bumped to 0.19.0.
  • Six user-facing integration files updated to reference version 0.19.0:
    .github/workflows/aqueduct-plan.yml, .github/workflows/aqueduct-apply.yml,
    ci/gitlab/aqueduct.gitlab-ci.yml, .pre-commit-hooks.yaml,
    docs/api-reference.md, docs/introduction.md.
  • ROADMAP.md status line updated to v0.19.0 released.

This release closes every open security finding from the Phase 4 engineering assessment, bringing pg_aqueduct to zero open CVEs and zero open critical or high security issues. The headline change is that consumer view bodies are now validated as a single SELECT statement before any database call is made — meaning a misconfigured or maliciously crafted migration file containing DDL like CREATE TABLE or DROP TABLE can never reach the database, even if it somehow made it into your migrations directory. This is an important defence-in-depth layer for teams running Aqueduct in automated CI/CD pipelines.

Two additional critical security fixes ship in this version: the CloudNativePG preview client now validates TLS certificates by default (previously it silently accepted any certificate), and the age secret backend now validates the identity file path for both path traversal and flag injection before passing it to the age subprocess. Together with the consumer SQL validation, these three fixes mean pg_aqueduct meets the full OWASP A03 (Injection) and A02 (Cryptographic Failures) controls relevant to its operation.

Beyond security, this version improves day-to-day operational quality: aqueduct plan --fail-on-drift now correctly detects drift in consumer views and source tables (not just stream tables), aqueduct destroy auto-migrates a stale catalog before querying it, and several documentation files and CI template files that had stale version pins are updated to match the current release. A new .github/SECURITY.md establishes the project's vulnerability reporting policy.

Added

  • validate_consumer_sql_is_single_select() is now called immediately before every
    CREATE OR REPLACE VIEW in the executor (ManageConsumerView and SwapConsumerViews
    arms). A migration file containing injected DDL is rejected with
    AqueductError::UntrustedSqlBody before any database call is made (SEC-1).
  • Integration test consumer_sql_injection_rejected_at_apply confirms that a consumer
    migration with a CREATE TABLE body is rejected at the executor level (SEC-1).
  • CNPG preview HTTP client now validates TLS certificates by default. Set
    CNPG_INSECURE_SKIP_VERIFY to skip verification in development environments only.
    Emits a warn! when the env var is set (SEC-2).
  • Unit test cnpg_tls_default_does_not_skip_verification confirms the env-var gate
    logic (SEC-2).
  • age secret backend now validates the identity file path (from SOPS_AGE_KEY_FILE /
    AGE_KEY_FILE) via validate_secret_path() and rejects paths starting with -
    (flag-injection guard). Unit tests age_identity_file_traversal_rejected and
    age_identity_file_flag_injection_rejected cover both rejection cases (SEC-3).
  • .github/SECURITY.md: security policy with contact address, response SLA, supported
    version matrix, and a summary of implemented controls (L-5).
  • docs/security.md: new sections covering consumer SQL injection protection,
    CNPG TLS configuration, and age identity file security (SEC-1, SEC-2, SEC-3).

Changed

  • aqueduct plan --fail-on-drift: drift count now includes source_deltas and
    consumer_deltas in addition to deltas, matching the already-correct status
    implementation. Previously, consumer-layer and source-layer drift was silently
    ignored (M-2).
  • aqueduct destroy: changed from connect() to connect_and_migrate() so a stale
    catalog schema is auto-migrated before destroy queries it (M-5).
  • release.yml: removed global FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 env var — all
    actions in the workflow use modern Node.js runtimes that do not require this
    workaround (M-6). Added just gen-docs step to regenerate docs/api-reference.md
    on every release tag.
  • ingest.rs test helpers: fs::write(...).unwrap() replaced with descriptive
    .expect("...") calls so a failing CI write produces a clear error message (M-7).
  • fmt.rs: renamed KNOWN_FRONTMATTER_KEYS to KNOWN_DIRECTIVE_KEYS with updated
    doc comment explaining it is used to skip already-emitted keys, not enforce ordering
    (L-1).
  • main.rs: replaced stale // L7 (v0.14) tracking comment with descriptive
    per-platform CI env var documentation (L-3).
  • Cargo.toml: replaced stale # DEP-1 (v0.14): pre-declare serde_yaml comment with
    # YAML serialization for plan/status/diff --format yaml output. (L-4).
  • Six user-facing integration files updated to ref...
Read more

v0.18.0

Choose a tag to compare

@github-actions github-actions released this 20 May 18:22

This release closes every open security finding from the Phase 4 engineering assessment, bringing pg_aqueduct to zero open CVEs and zero open critical or high security issues. The headline change is that consumer view bodies are now validated as a single SELECT statement before any database call is made — meaning a misconfigured or maliciously crafted migration file containing DDL like CREATE TABLE or DROP TABLE can never reach the database, even if it somehow made it into your migrations directory. This is an important defence-in-depth layer for teams running Aqueduct in automated CI/CD pipelines.

Two additional critical security fixes ship in this version: the CloudNativePG preview client now validates TLS certificates by default (previously it silently accepted any certificate), and the age secret backend now validates the identity file path for both path traversal and flag injection before passing it to the age subprocess. Together with the consumer SQL validation, these three fixes mean pg_aqueduct meets the full OWASP A03 (Injection) and A02 (Cryptographic Failures) controls relevant to its operation.

Beyond security, this version improves day-to-day operational quality: aqueduct plan --fail-on-drift now correctly detects drift in consumer views and source tables (not just stream tables), aqueduct destroy auto-migrates a stale catalog before querying it, and several documentation files and CI template files that had stale version pins are updated to match the current release. A new .github/SECURITY.md establishes the project's vulnerability reporting policy.

Added

  • validate_consumer_sql_is_single_select() is now called immediately before every
    CREATE OR REPLACE VIEW in the executor (ManageConsumerView and SwapConsumerViews
    arms). A migration file containing injected DDL is rejected with
    AqueductError::UntrustedSqlBody before any database call is made (SEC-1).
  • Integration test consumer_sql_injection_rejected_at_apply confirms that a consumer
    migration with a CREATE TABLE body is rejected at the executor level (SEC-1).
  • CNPG preview HTTP client now validates TLS certificates by default. Set
    CNPG_INSECURE_SKIP_VERIFY to skip verification in development environments only.
    Emits a warn! when the env var is set (SEC-2).
  • Unit test cnpg_tls_default_does_not_skip_verification confirms the env-var gate
    logic (SEC-2).
  • age secret backend now validates the identity file path (from SOPS_AGE_KEY_FILE /
    AGE_KEY_FILE) via validate_secret_path() and rejects paths starting with -
    (flag-injection guard). Unit tests age_identity_file_traversal_rejected and
    age_identity_file_flag_injection_rejected cover both rejection cases (SEC-3).
  • .github/SECURITY.md: security policy with contact address, response SLA, supported
    version matrix, and a summary of implemented controls (L-5).
  • docs/security.md: new sections covering consumer SQL injection protection,
    CNPG TLS configuration, and age identity file security (SEC-1, SEC-2, SEC-3).

Changed

  • aqueduct plan --fail-on-drift: drift count now includes source_deltas and
    consumer_deltas in addition to deltas, matching the already-correct status
    implementation. Previously, consumer-layer and source-layer drift was silently
    ignored (M-2).
  • aqueduct destroy: changed from connect() to connect_and_migrate() so a stale
    catalog schema is auto-migrated before destroy queries it (M-5).
  • release.yml: removed global FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 env var — all
    actions in the workflow use modern Node.js runtimes that do not require this
    workaround (M-6). Added just gen-docs step to regenerate docs/api-reference.md
    on every release tag.
  • ingest.rs test helpers: fs::write(...).unwrap() replaced with descriptive
    .expect("...") calls so a failing CI write produces a clear error message (M-7).
  • fmt.rs: renamed KNOWN_FRONTMATTER_KEYS to KNOWN_DIRECTIVE_KEYS with updated
    doc comment explaining it is used to skip already-emitted keys, not enforce ordering
    (L-1).
  • main.rs: replaced stale // L7 (v0.14) tracking comment with descriptive
    per-platform CI env var documentation (L-3).
  • Cargo.toml: replaced stale # DEP-1 (v0.14): pre-declare serde_yaml comment with
    # YAML serialization for plan/status/diff --format yaml output. (L-4).
  • Six user-facing integration files updated to reference version 0.18.0:
    .github/workflows/aqueduct-plan.yml, .github/workflows/aqueduct-apply.yml,
    ci/gitlab/aqueduct.gitlab-ci.yml (also fixes archive name pattern to match
    the release pipeline: aqueduct-{version}-{os}-{arch}.tar.gz),
    .pre-commit-hooks.yaml, docs/api-reference.md, docs/introduction.md (CI-1).
  • version-lint CI job extended to check nine files: README.md,
    docs/installation.md, docs/api-reference.md, docs/introduction.md,
    ci/gitlab/aqueduct.gitlab-ci.yml, .pre-commit-hooks.yaml,
    .github/workflows/aqueduct-plan.yml, .github/workflows/aqueduct-apply.yml,
    and ROADMAP.md. Fails if any disagrees with the workspace Cargo.toml version
    (CI-1, L-6).
  • ROADMAP.md status line updated to v0.18.0 released (L-6).
  • Workspace version bumped to 0.18.0.

v0.17.0

Choose a tag to compare

@github-actions github-actions released this 20 May 13:05
14be9ef

Added

  • --patroni-endpoint <url> flag on aqueduct apply. Between each migration step,
    the CLI calls GET <endpoint>/master; a non-200 response marks the migration
    interrupted and aborts (DOC-2 / v0.17-A).
  • aqueduct_core::ha module: HaBackend enum, detect_ha_backend(),
    check_still_primary() (uses pg_is_in_recovery()), and check_patroni_primary()
    HTTP check (DOC-2 / v0.17-A).
  • Between-step primary re-check via SELECT pg_is_in_recovery() in the executor.
    On failover, migration is marked interrupted (v0.17-A).
  • Per-step JSON events emitted to stderr: step_start, step_complete, step_failed
    with step_index, step_type, migration_id, project, and duration_ms fields
    (v0.17-B). Documented in docs/cli-events-schema.json.
  • aqueduct audit subcommand: lists recent migrations with step counts, error messages,
    and durations. Supports --format table|json|yaml and --limit N (v0.17-B).
  • aqueduct.migration_history SQL view (catalog v8): joins aqueduct.migrations and
    aqueduct.migration_steps for per-step audit queries (v0.17-B).
  • aqueduct_core::catalog::LIST_MIGRATIONS_FOR_AUDIT_SQL query constant (v0.17-B).
  • Prometheus metrics endpoint behind --features metrics on aqueduct-cli.
    Pass --metrics-addr 0.0.0.0:9090 to expose /metrics during apply (v0.17-B).
  • Catalog schema v8: adds 'interrupted' to the migrations.status check constraint
    and a partial index for interrupted rows. Includes auto-migration from v7 (v0.17-A).
  • GitHub-native build provenance attestation in release workflow via
    actions/attest-build-provenance@v2. Verify with gh attestation verify (v0.17-C).
  • release-verify CI job: downloads the linux-amd64 release archive, verifies its
    SHA256 checksum, and confirms aqueduct --version matches the tag (v0.17-C).
  • Version linting CI step (version-lint job): fails if README.md status banner,
    docs/installation.md, and Cargo.toml workspace version disagree (v0.17-D).
  • just publish-dry-run recipe for dry-run crates.io publish verification (v0.17-C).

Changed

  • Catalog schema bumped from v7 to v8 (CATALOG_SCHEMA_VERSION = 8).
  • aqueduct-core and aqueduct-testkit Cargo.toml: publish = true;
    aqueduct-cli: publish = false (v0.17-C).
  • docs/ha-operations.md rewritten to accurately describe implemented behaviors:
    check_still_primary(), Patroni /master check, per-step events, aqueduct audit,
    and the recovery runbook. Speculative system_identifier/timeline_id content removed
    (v0.17-D).
  • docs/installation.md: updated version example to 0.17.0, MSRV to 1.88,
    added build provenance verification section (v0.17-C/D).
  • ci.yml security-audit job: removed --ignore RUSTSEC-2025-0052 (httpmock replaced).
  • httpmock replaced with wiremock in aqueduct-core dev-dependencies; all five
    HTTP mock tests updated to the async wiremock API (v0.17-D).

v0.16.0

Choose a tag to compare

@github-actions github-actions released this 20 May 12:03
43c7f81

Added

  • OutputMode enum (Human, Json, Yaml, Quiet, Porcelain) and
    OutputEmitter struct in aqueduct-cli::output; global --quiet and
    --porcelain flags are now routed through the emitter (ERG-1, M11).
  • serde_yaml workspace dependency; YAML output in plan, status, and diff
    subcommands now uses serde_yaml::to_string instead of hand-rolled format strings,
    correctly escaping quotes, colons, and newlines (ERG-3).
  • just gen-docs recipe that regenerates docs/api-reference.md from
    aqueduct --help output; stale references to --patroni-endpoint, --timeout,
    --lock-timeout, and --no-cost removed (ERG-4, M3).
  • Binary CLI tests (assert_cmd) for every subcommand's exit codes and --help
    output: quiet_suppresses_decorative_output, porcelain_outputs_key_value_only,
    yaml_escapes_quotes_and_newlines, plan_help_documents_fail_if_changed,
    apply_help_documents_dry_run, status_help_documents_fail_on_drift,
    diff_help_documents_fail_on_drift, destroy_help_documents_dry_run,
    rollback_help_documents_dry_run, plan_missing_dsn_exits_2,
    validate_exits_0_on_valid_files, validate_differential_ivm_unsupportable_fails,
    validate_format_json_produces_valid_json, version_flag_outputs_semver (TEST-2).
  • --poll-interval flag on aqueduct status (PERF-2).
  • Spec-hash caching in status --watch: migration files are only reloaded when any
    .sql mtime changes; live state always polled fresh (PERF-2).
  • postgres_version_matrix_min_supported integration test verifying the full
    create/apply/status cycle on PostgreSQL 18+ (H10).
  • CONTRIBUTING.md documenting Docker/Testcontainers prerequisites, just recipes,
    integration test environment variables, PG version matrix, coding standards, and
    a step-by-step guide for adding cookbook recipes.

Changed

  • fmt::format_migration now returns Result<Option<String>> and propagates IO
    errors; aqueduct fmt prints a diagnostic when a file cannot be read instead of
    silently treating it as empty (L2).
  • CANONICAL_KEY_ORDER constant in aqueduct-core::fmt renamed to
    KNOWN_FRONTMATTER_KEYS to reflect its actual role (L3).
  • TestDb::connection_string field visibility narrowed from pub to pub(crate)
    to prevent external crates from depending on the internal representation (L14).
  • docs/security.md: AWS, GCP, and Vault secret backend rows updated from
    "Planned (v0.12)" to "Implemented"; injected SQL trust-boundary note updated to
    describe the CatalogSchema newtype introduced in v0.14.
  • docs/api-reference.md regenerated from aqueduct --help output.

Fixed

  • aqueduct status --format yaml now produces valid YAML that passes
    serde_yaml::from_str round-trip including special characters.
  • aqueduct plan --format yaml correctly escapes project names containing
    quotes, colons, and newlines.

v0.15.0

Choose a tag to compare

@github-actions github-actions released this 20 May 11:18
d09bcf1

Added

  • Saga-style compensating steps: CreateStreamTable and DropStreamTable write a
    compensating action to aqueduct.ddl_log before executing DDL, enabling crash
    recovery via --resume.
  • --force-retry <step-index> and --force-skip <step-index> flags on
    aqueduct apply to advance past ambiguous steps (require --yes).
  • CatalogSchema newtype validating catalog schema names at parse time, rejecting SQL
    injection markers, non-identifier characters, leading digits, and reserved system
    schema names (pg_catalog, information_schema, all pg_* prefixed names).
  • ExecutionContext enum with Apply, Rollback, Promote, and DryRun variants
    replacing the optional builder pattern; named constructors for_apply(),
    for_rollback(), and for_promote().
  • StartBlueGreenDeployment plan step inserting a row into
    aqueduct.blue_green_deployments with status = 'active' as the first post-lock
    step.
  • All SwapConsumerViews steps wrapped in a single BEGIN ... COMMIT block for an
    all-or-nothing consumer view swap.
  • Real RLS policy capture via pg_policies / pg_class.relrowsecurity before
    DropStreamTable; RecreatePolicy step restores policies after table recreation;
    failures are non-fatal and logged to aqueduct.ddl_log.
  • rebuild-may-affect-rls lint warning when a FULL-mode table has RLS enabled.
  • Batch convergence polling: WaitForConvergence issues a single
    SELECT ... WHERE table_name = ANY($2) per poll interval instead of one query per
    node.
  • convergence_poll_interval_ms (default 500 ms) and convergence_timeout_secs
    (default 300 s) options in BuildPlanOptions.
  • validate_migration_files_diagnostic and validate_dag_diagnostic public functions
    returning structured DiagnosticSet entries with file paths, error codes, and
    severity levels.
  • aqueduct validate --format json output matching the lint JSON schema.
  • Catalog schema v7: aqueduct.ddl_log gains migration_id bigint and
    compensating_sql text; aqueduct.blue_green_deployments gains
    rolled_back_at timestamptz and an updated bg_status_check constraint accepting
    'rolled_back'.

Changed

  • Executor writes status = 'swapped' after the atomic consumer view swap and
    status = 'retired' after RetireBlueSchema.
  • Integration tests now target postgres:18-alpine exclusively (PostgreSQL 18+
    required by pg_trickle).

v0.14.0

Choose a tag to compare

@github-actions github-actions released this 20 May 09:49
9ac94b7

Status: Released

All roadmap items for v0.14 "Phase 17 — Failure Safety, CI/CD Trustworthiness &
Security Hardening" are implemented and tested.

Highlights

  • Correctness fixes (CORR-1 through CORR-8): Seven correctness bugs were found
    and fixed. Progress is now preserved on recoverable failures (CORR-1); the heartbeat
    correctly signals the main executor loop via a tokio::sync::watch channel when the
    advisory lock is lost (CORR-3); promotion filters the destination state by project
    (CORR-4); the promote command migrates the catalog before use and records spec_jsonb
    (CORR-5); consumer view drops now delete the catalog row so drift counts stay
    accurate (CORR-7); and status drift counting now sums stream, source, and consumer
    deltas (CORR-8).

  • Catalog v6: The catalog schema is bumped to version 6. The new migration adds
    the pgtrickle_mock.scheduler_state table used by integration tests to assert that
    pause_scheduler / resume_scheduler calls are routed correctly (TEST-3).

  • Security hardening (SEC-1 – SEC-3):

    • SEC-1: Keyword-value DSNs (e.g. host=... password=secret) are now also
      checked for plaintext passwords; previously only URI-style DSNs were validated.
    • SEC-2: Consumer SQL bodies are validated to be a single SELECT statement
      before the view is created, preventing DDL injection via migration files.
    • SEC-3: The read-only helper now issues BEGIN READ ONLY before setting
      SET LOCAL statement_timeout, so the timeout is correctly scoped to the
      transaction. The sanitize_statement_timeout allowlist prevents injection via
      the timeout argument.
  • Ergonomics (ERG-2): aqueduct destroy now exits with code 2 on error (via
    anyhow::bail!) instead of calling std::process::exit(1) directly, ensuring the
    error message is routed through the standard error handler.

  • Lint fixes (L1, L7):

    • L1: The DSN-redaction regex in the CLI is now compiled once via LazyLock
      instead of being recompiled on every call.
    • L7: CI environment variables (GITHUB_ACTIONS, CI, GITLAB_CI,
      CIRCLECI) are now checked for the string value "true" rather than merely being
      set, preventing false positive CI detection.
  • CI/CD fixes (CI-1 – CI-3):

    • CI-1: The composite plan and apply GitHub Actions now map the runner
      OS/architecture to the correct release archive suffix (linux-amd64,
      linux-arm64, macos-arm64, macos-amd64, windows-amd64).
    • CI-2: The action-smoke workflow now invokes the composite actions end-to-end
      using a locally-packaged archive (via the new local-archive input). The
      migration file path was corrected to migrations/streams/event_count.sql.
    • CI-3: mdbook test is now a blocking step (removed continue-on-error: true).
      The docs-lint job no longer checks for a non-existent export subcommand, and
      the subcommand reference check now fails the job if any documented subcommand is
      missing from the binary.
  • Test infrastructure (TEST-3): The pgtrickle_mock schema now includes a
    scheduler_state table. The mock pause_scheduler / resume_scheduler functions
    insert and delete rows in this table, enabling integration tests to assert scheduler
    state transitions. Nine new integration tests cover all CORR, SEC, and TEST items.

  • pg_version in JSON output (M15): aqueduct status --format json now includes
    a "pg_version" field.

Migration notes

  • Catalog: Run aqueduct init (or aqueduct apply) to apply the v5→v6 catalog
    migration. The migration is fully backward-compatible.

  • Consumer SQL validation: If you have consumer migration files whose SQL body is
    not a pure SELECT statement, aqueduct apply will now reject them with error
    code 1307. Update those files to use a single SELECT.


v0.13.0

Choose a tag to compare

@github-actions github-actions released this 20 May 08:19

Status: Released

All roadmap items for v0.13 "Phase 16 — Blue/Green End-to-End, IMMEDIATE Mode &
Advanced Features" are implemented and tested.

Highlights

  • Blue/green migrations (--strategy blue-green): The planner now generates the
    full blue/green step sequence — CreateGreenSchema, CreateStreamTableInGreen,
    WaitForConvergence (with real pg_trickle polling), SwapConsumerViews, and
    RetireBlueSchema — for topology-restructuring diffs.

  • IMMEDIATE refresh mode: RefreshMode::Immediate is parsed from
    -- @aqueduct:refresh_mode = "IMMEDIATE", classified, stored, and correctly handled
    in rebuild paths via PauseImmediate/ResumeImmediate steps. The new
    --no-immediate-downgrade flag rejects plans that would temporarily degrade an
    IMMEDIATE table.

  • Pre/post migration hooks: SQL statements from [apply.hooks] pre and post
    in aqueduct.toml are emitted as RunHook steps. The pre-hook fires immediately
    after LockDag; the post-hook fires just before RecordSnapshot.

  • Immutable plan artifacts (--out / --plan): aqueduct plan --out plan.json
    writes the plan (with a SHA-256 spec hash) to a file. aqueduct apply --plan plan.json
    loads and executes the pre-computed plan, validating the spec hash to reject stale
    artifacts.

  • Step observability (migration_steps): The catalog is bumped to v5, adding
    aqueduct.migration_steps. Every executed step writes a running row at start and
    a done (or failed) row at completion, enabling audit and replay.

  • Import baseline snapshot: aqueduct import now calls ensure_catalog_current
    and records version 1 in aqueduct.dag_versions. A subsequent aqueduct plan
    returns an empty plan.

  • Real preview backends: The create_preview_cnpg and create_preview_neon stubs
    are replaced with real HTTP implementations using the CNPG Kubernetes API and the
    Neon management API respectively.

  • Rollback classification (--within-window / --accept-data-loss): The rollback
    command now classifies each change as Safe, PointInTime, or DataLoss based on
    when the version was applied relative to the pg_trickle WAL retention window.

  • JSON schema versioning (schema_version: 1): All structured JSON events emitted
    by the CLI now include "schema_version": 1. The schema is published at
    docs/cli-events-schema.json.


v0.11.0

Choose a tag to compare

@github-actions github-actions released this 19 May 21:11

Status: Released

All roadmap items for v0.11 "Phase 14 — Diagnostic Quality, CLI Surface &
Documentation Correctness" are complete. This release focuses on making the CLI
output trustworthy, the error messages actionable, and the documentation honest.

What's new

Diagnostic Infrastructure

  • Q-08: New unified Diagnostic type (aqueduct_core::diagnostic) replaces
    the separate ValidationError, LintDiagnostic, and ad-hoc parse error strings.
    DiagnosticSet collects diagnostics with file/line/column context. Both the
    validate and lint subsystems now emit DiagnosticSet results.

  • Q-04: Stable numeric error codes (error_code() method on AqueductError).
    Error categories: 1000–1099 database, 1100–1199 configuration, 1200–1299 planning,
    1300–1399 execution, 1400–1499 catalog. Codes are stable across releases.

  • Q-03: ManageWalSlot plan step uses typed WalSlotAction enum (Create/Drop)
    instead of a free-form String. Eliminates the "Unknown action" dead code path.

CLI Surface Correctness

  • U-02: Fixed aqueduct plan exit semantics. Previously exited 1 for any non-empty
    plan. Now exits 0 by default; exits 1 only when --fail-if-changed or
    --fail-on-drift is explicitly set. CI pipelines that just want to see a plan
    without failing now work correctly.

  • U-03: New --fail-on-drift flag for aqueduct diff. Exits 1 when any delta is
    non-Unchanged, exits 0 for a clean diff.

  • U-05: executor.execute() now returns ExecutionResult { migration_id, dag_version }
    instead of a bare u64. The apply_complete JSON event on stderr now emits both
    migration_id (aqueduct.migrations row id) and dag_version (bigserial) as separate
    fields.

  • U-06/SEC-01: Password redaction for all connection strings displayed in error
    messages. redact_dsn() replaces passwords with *** in both URL-form and
    keyword-value DSNs.

  • U-10: YAML output format for aqueduct status (use --format yaml).

Read-Only Safety (D-03/SEC-08)

  • aqueduct plan and aqueduct status now use connect_read_only(), which issues
    SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLY immediately after connecting.
    These commands cannot accidentally mutate data.

Documentation Correctness

  • D-01: README updated to v0.11.0 status and feature list.

  • D-04/D-05: Blue/green migration strategy (still planned for v0.13) is now clearly
    marked as [PLANNED] in docs/api-reference.md, docs/ha-operations.md, and the
    apply action YAML. The --strategy blue-green flag in the apply action is disabled
    (no-op) pending the v0.13 implementation.

  • D-07: Unimplemented features (YAML output, IMMEDIATE mode, real secret backends)
    moved from [Released] to [Planned] sections in CHANGELOG.

  • D-09/SEC-02: docs/security.md now clearly marks AWS/GCP/Vault secret backends
    as "Planned (v0.12)". Only env, sops, and age are implemented today.

  • D-11: docs/ha-operations.md maintenance window documentation updated to remove
    blue/green from the implemented maintenance_window_applies_to values.

  • D-12: Example workflow version pins in aqueduct-plan.yml and aqueduct-apply.yml
    updated from @v0.4.0 to @v0.11.0.

Code Quality

  • Q-01: Step registry contract test — exhaustive match on all PlanStep variants
    ensures every variant has a non-empty description(). Adding a new variant without
    updating description() is now a compile error.

  • Q-02: Typed PlanExecutor constructors: for_apply(), for_rollback(),
    for_promote(), for_dry_run(). The generic new() constructor is still available.

  • Q-06: Typed DAG state wrappers: DesiredDagState, LiveDagState,
    RecordedDagState newtypes wrapping DagState. Prevents mixing state kinds in
    function signatures.

  • Q-09: lib.rs exports CLI_VERSION = env!("CARGO_PKG_VERSION") as the single
    source of version truth. All commands use env!("CARGO_PKG_VERSION") directly.

CI Improvements

  • M-13: docs-build job in CI runs mdbook build and mdbook test on every push.

  • U-01/D-02: docs-lint job in CI builds the release binary and verifies that
    every subcommand responds to --help, and that the binary --version matches
    Cargo.toml.

  • New docs-cli recipe in justfile generates a Markdown CLI reference from
    --help output.

Security

  • SEC-07: New docs/roles.sql contains SQL grant templates for the four aqueduct
    roles: aqueduct_planner, aqueduct_applier, aqueduct_preview, aqueduct_destroy.

v0.10.0

Choose a tag to compare

@github-actions github-actions released this 19 May 20:28
6e00ac5

Status: Released

All roadmap items for v0.10 "Phase 13 — Safety Contract Repair & Multi-Project
Isolation" are complete. This release repairs the safety contract (lock lifecycle,
resume, heartbeat, scheduler pause ordering) and establishes the multi-project
isolation invariant required for shared PostgreSQL deployments.

What's new

Critical Correctness Fixes

  • C-01 / C-11: Consumer-only plans are no longer silently skipped as no-ops.
    PlanSummary now tracks consumer_creates, consumer_drops, and consumer_alters
    separately; is_empty() and total_changes() include consumer deltas. All renderer
    totals updated.

  • C-03: RecordSnapshot now uses query_one with RETURNING version to capture
    the actual bigserial value assigned by the database instead of recording the planned
    version number.

  • C-04: Backfill steps are documented as "Triggered by pg_trickle on table
    creation — no explicit wait" in renderers and log messages.

  • C-05: read_live_state() now populates sources by deserialising the latest
    spec_jsonb from aqueduct.dag_versions instead of always returning vec![].

  • C-06 / C-07: read_live_state() and read_live_consumers() both accept an
    optional project parameter and filter results via the new
    aqueduct.stream_table_ownership table. destroy_project verifies ownership before
    dropping any table; use --force-unowned to bypass.

  • C-09: check_pgtrickle_version() probes to_regprocedure(...) before calling
    the function to avoid a database error when pg_trickle is absent.

  • C-10: classify_delta() no longer panics on missing desired/actual in the
    AlterQuery arm; returns Rebuild instead.

  • C-12: read_live_state() project filter enables diff --from last-applied and
    diff --from desired modes.

Safety & Idempotency Fixes

  • S-01 / S-02: Failed migrations are now marked recoverable_failure and can be
    resumed. On resume, LockDag always re-executes — DDL steps are only skipped for
    steps already confirmed complete in the checkpoint.

  • S-03: Scheduler pause now happens inside the LockDag executor arm, after the
    lock is acquired, preventing a pre-lock pause from affecting other projects.

  • S-04: Heartbeat SQL is now holder-bound
    (WHERE project = $1 AND holder = $2). Zero rows affected is detected and logged as
    a lock-stolen warning.

  • S-05: The project lock is released on all exit paths (success and error) via an
    explicit cleanup block after the step loop.

  • S-06: Step progress checkpoint writes are now fatal — a failed write returns an
    error rather than silently continuing.

  • S-07: aqueduct apply --dry-run uses a plain connection (no catalog migration)
    to avoid upgrading the schema during previews.

  • S-10: DROP TABLE fallback in destroy and the executor no longer uses
    CASCADE. Dependent objects must be dropped manually; use --force-cascade to
    opt in.

  • S-11: aqueduct rollback now enforces the --accept-data-loss flag when the
    rollback plan contains rebuild-class steps.

  • S-12: First-time stream table creations increment create_count and not
    rebuild_count, preventing false-positive maintenance-window blocking on initial
    deployments.

  • S-13: Lock holder string is now aqueduct/{version}/{hostname}/{pid}/{ts}
    making concurrent processes distinguishable.

  • S-14: Scheduler pause failure is now a fatal error by default.

Catalog Schema v3

  • Added aqueduct.stream_table_ownership(project, schema_name, table_name, managed_since)
    table with a PRIMARY KEY (schema_name, table_name) constraint.
  • status_check constraint on aqueduct.migrations now includes recoverable_failure.
  • Auto-migration from v2 to v3 applied on connect_and_migrate.

New Tests (6)

  • test_consumer_only_apply_end_to_end: consumer view create with no stream table changes.
  • test_concurrent_apply_race: two applies, second gets LockContention.
  • test_stale_plan_detection: plan from_version mismatch vs DB version.
  • test_two_project_isolation: destroy project A, project B tables intact.
  • test_resume_skips_non_safety_steps: LockDag always at index 0.
  • test_heartbeat_lock_is_holder_bound: wrong holder returns 0 rows.