Releases: trickle-labs/pg-aqueduct
Release list
v0.20.0
This release delivers the three high-priority architectural improvements identified in the Phase 4 engineering assessment: typed executor configuration, full multi-tenant catalog parameterisation, and automated crash recovery via the compensating-step registry.
The ExecutionContext struct replaces the optional builder pattern for the two safety-critical executor fields (desired_state and connection_string), making it a compile error to omit them. All command handlers (apply, rollback, promote) have been updated accordingly.
Every catalog SQL constant now accepts a CatalogSchema parameter and substitutes the schema name via for_schema(). The validate_catalog_schema_not_overridden() stop-gap from v0.19 is removed. aqueduct init --schema my_catalog creates all catalog tables in my_catalog; the multi-tenant isolation integration test verifies that two projects on the same database with different catalog schemas have fully independent version histories.
The --resume flow now queries aqueduct.ddl_log for rows in status = 'running' before advancing to the checkpoint. For each such row the compensating SQL is executed and the row is updated to status = 'compensated'. A new integration test covers the crash-after-DDL, before-RecordSnapshot scenario.
The OutputEmitter is now stored as a process-wide singleton via OnceLock, initialised in main before command dispatch and accessible via output::emitter(). The init command is the first to use the emitter for its success messages.
The httpmock dependency was replaced with wiremock in v0.17, eliminating the last cargo audit --deny warnings finding (RUSTSEC-2025-0052).
The catalog schema version advances from 8 to 9, adding a status column to aqueduct.ddl_log used by compensating-step recovery.
Added
- Typed
ExecutionContextstruct (ARCH-2):ExecutionContextstruct with required
desired_stateandconnection_stringfields replaces the optional builder pattern.
Omitting either field is now a compile error.for_apply(),for_rollback(), and
for_promote()constructors updated. - Full catalog schema parameterisation (ARCH-1):
CatalogSchemathreaded through
every catalog SQL function.ensure_catalog_current(),connect_and_migrate(),
read_live_state(),get_latest_dag_version(),detect_extension_installed(),
read_ddl_log(),import_from_live(),compute_promotion_plan(),
validate_source_clean(), anddestroy_project()all accept&CatalogSchema.
validate_catalog_schema_not_overridden()stop-gap removed. - Multi-tenant catalog isolation test:
test_multi_tenant_catalog_isolationverifies
that two catalogs (tenant_a,tenant_b) on the same database have independent
dag_versionstables with no cross-contamination. - Automated compensating-step recovery (CORR-2):
find_resume_step()queries
aqueduct.ddl_logforstatus = 'running'rows and executes theircompensating_sql
viabatch_executebefore advancing to the resume checkpoint. Rows are updated to
status = 'compensated'on success. COMPLETE_COMPENSATING_STEP_SQLandMARK_COMPENSATING_APPLIED_SQLconstants in
catalog.rsfor the two-phase compensating-step lifecycle.- CORR-2 crash recovery integration test:
test_corr2_compensating_step_recovery
inserts a syntheticstatus = 'running'ddl_log row and verifies that--resumemoves
it out of therunningstate. - Catalog schema version 9:
CATALOG_SCHEMA_VERSIONadvances from 8 to 9. The v8→v9
migration adds astatus TEXT NOT NULL DEFAULT 'complete'column toaqueduct.ddl_log
and backfills existing rows. OutputEmitterprocess-wide singleton (M-1):output::init_emitter(mode)stores
the emitter in astatic OnceLock<OutputEmitter>;output::emitter()returns the
global reference.main.rscallsinit_emitterbefore dispatch. Theinitcommand
uses the emitter for its success messages, replacing directprintln!calls.
Changed
for_schema(sql, schema)now substitutes= 'aqueduct'string-literal comparisons
andCREATE/DROP SCHEMA IF NOT EXISTS aqueductstatements in addition to the
aqueduct.dot-prefix references.DestroyOptionsgains acatalog_schema: CatalogSchemafield.- All
cargo check --testscall sites forread_live_state,get_latest_dag_version,
detect_extension_installed,read_ddl_log,import_from_live,
compute_promotion_plan,validate_source_clean, andDestroyOptions::newupdated
throughout CLI commands and integration tests. - Workspace version bumped
0.19.0→0.20.0.
Fixed
executor.rs:import_from_livewas callingread_live_state(client, None)(2-arg form)
instead of the 3-argread_live_state(client, None, catalog_schema). Fixed.- Extra trailing semicolons (
await?;;) incommands/promote.rsandcommands/status.rs
introduced by the multi-replace pass. Removed.
v0.19.0
This release eliminates every structural correctness bug identified in the Phase 4 engineering assessment, makes the blue/green view swap fully atomic, and ensures every known correctness regression path is covered by a test that would catch it.
The headline change is that the deployment status row update in SwapConsumerViews is now executed inside the surrounding BEGIN/COMMIT block, eliminating the window where consumer views point at the green schema while the deployment row still shows status = 'active'. If the transaction is rolled back for any reason, both the view assignments and the deployment row revert together.
The heartbeat task is now supervised by a JoinHandle-based wrapper. If the heartbeat task panics — something that could previously produce a silent hang — the executor immediately receives a LockLost signal, stops execution, and surfaces the failure. This closes the last known path to a silent runaway executor.
The mock pg_trickle DDL gains a pgtrickle.paused_nodes table so integration tests can assert precisely which nodes were paused during DDL and that the scheduler was fully resumed after apply or after a failure. Two new test-kit helpers (assert_scheduler_idle and assert_scheduler_paused_for) make these assertions one-liners.
Four executor and CLI quality fixes round out the release: the WaitForConvergence poll query is now guarded by a statement_timeout to prevent a hung pg_trickle query from holding the loop past the configured deadline; status --watch applies exponential backoff on repeated connection failures; the EXPLAIN trade-off in cost.rs is documented with a code comment; and aqueduct init and aqueduct apply now return an explicit NotYetImplemented error when catalog_schema is set to anything other than "aqueduct", preventing silent data mixing in multi-tenant deployments until full schema parameterisation ships in v0.20.
All ten failure-mode test cases identified in the Phase 4 assessment are now present and passing.
Added
- Blue/green atomicity (ARCH-3):
SWAP_BLUE_GREEN_SQL(theblue_green_deployments
status row update) is now executed inside theSwapConsumerViewstransaction. A failed
mid-swap now rolls back both view assignments and the deployment status atomically.
Integration testswap_consumer_views_atomicityverifies this. - Heartbeat panic supervisor (CORR-3):
tokio::spawn(run_heartbeat(...))is wrapped in
aJoinHandle-based supervisor that callslock_lost_tx.send(true)if the heartbeat
task panics. Integration testheartbeat_panic_signals_lock_lossverifies the signal. - Observable scheduler mock (TEST-3):
mock_pgtrickle.rsgains a
pgtrickle.paused_nodes(node_name text PRIMARY KEY, paused_at timestamptz)table.
pause_schedulerinserts into it;resume_schedulerdeletes from it. - Test-kit helpers:
assert_scheduler_idle(client)and
assert_scheduler_paused_for(client, nodes)added toaqueduct-testkit. Integration
testsmock_scheduler_records_pauseandmock_scheduler_idle_after_applydemonstrate
usage. statement_timeoutguard forWaitForConvergence(M-9): the pollSELECTis
wrapped inBEGIN READ ONLY; SET LOCAL statement_timeout = '{timeout}ms'so a hung
pg_tricklequery cannot hold the loop open past the configured deadline. Integration
testwaitforconvergence_deadline_respectedverifies the deadline is enforced.- Exponential backoff in
status --watch(M-8): consecutive connection or poll
failures back off up to5 × intervalbefore retrying; the counter resets on the next
successful poll. - EXPLAIN trade-off comment (M-4):
cost.rs::estimate_rowsnow documents why
format!("EXPLAIN...")is intentional and safe. The EXPLAIN call is also wrapped in a
BEGIN READ ONLY; SET LOCAL statement_timeout = '5s'block. - Catalog schema stop-gap (ARCH-1 partial):
validate_catalog_schema_not_overridden
incatalog.rsreturnsAqueductError::NotYetImplemented { feature: "catalog_schema override" }
for any value other than"aqueduct". Called fromaqueduct initandaqueduct apply
to prevent silent data mixing until full multi-tenant support lands in v0.20.
Integration testcatalog_schema_override_rejected_until_implementedverifies this. - Failure-mode tests (Phase 4): all ten test cases identified in the assessment are
now present and passing:
consumer_sql_injection_rejected_at_apply,
heartbeat_panic_signals_lock_loss,
swap_consumer_views_atomicity,
plan_fail_on_drift_counts_consumer_deltas,
destroy_auto_migrates_catalog,
catalog_schema_override_rejected_until_implemented,
age_identity_file_traversal_rejected,
cnpg_preview_requires_valid_cert,
waitforconvergence_deadline_respected,
mock_scheduler_records_pause. AqueductError::NotYetImplemented { feature }variant added (error code 1308).cnpg_preview_requires_valid_certunit test inpreview.rsverifies that a
create_preview_cnpgcall to a plain-HTTP endpoint fails with a TLS/connection error
whenCNPG_INSECURE_SKIP_VERIFYis absent (SEC-2).
Changed
- Workspace version bumped to
0.19.0. - Six user-facing integration files updated to reference version
0.19.0:
.github/workflows/aqueduct-plan.yml,.github/workflows/aqueduct-apply.yml,
ci/gitlab/aqueduct.gitlab-ci.yml,.pre-commit-hooks.yaml,
docs/api-reference.md,docs/introduction.md. ROADMAP.mdstatus line updated tov0.19.0 released.
This release closes every open security finding from the Phase 4 engineering assessment, bringing pg_aqueduct to zero open CVEs and zero open critical or high security issues. The headline change is that consumer view bodies are now validated as a single SELECT statement before any database call is made — meaning a misconfigured or maliciously crafted migration file containing DDL like CREATE TABLE or DROP TABLE can never reach the database, even if it somehow made it into your migrations directory. This is an important defence-in-depth layer for teams running Aqueduct in automated CI/CD pipelines.
Two additional critical security fixes ship in this version: the CloudNativePG preview client now validates TLS certificates by default (previously it silently accepted any certificate), and the age secret backend now validates the identity file path for both path traversal and flag injection before passing it to the age subprocess. Together with the consumer SQL validation, these three fixes mean pg_aqueduct meets the full OWASP A03 (Injection) and A02 (Cryptographic Failures) controls relevant to its operation.
Beyond security, this version improves day-to-day operational quality: aqueduct plan --fail-on-drift now correctly detects drift in consumer views and source tables (not just stream tables), aqueduct destroy auto-migrates a stale catalog before querying it, and several documentation files and CI template files that had stale version pins are updated to match the current release. A new .github/SECURITY.md establishes the project's vulnerability reporting policy.
Added
validate_consumer_sql_is_single_select()is now called immediately before every
CREATE OR REPLACE VIEWin the executor (ManageConsumerViewandSwapConsumerViews
arms). A migration file containing injected DDL is rejected with
AqueductError::UntrustedSqlBodybefore any database call is made (SEC-1).- Integration test
consumer_sql_injection_rejected_at_applyconfirms that a consumer
migration with aCREATE TABLEbody is rejected at the executor level (SEC-1). - CNPG preview HTTP client now validates TLS certificates by default. Set
CNPG_INSECURE_SKIP_VERIFYto skip verification in development environments only.
Emits awarn!when the env var is set (SEC-2). - Unit test
cnpg_tls_default_does_not_skip_verificationconfirms the env-var gate
logic (SEC-2). agesecret backend now validates the identity file path (fromSOPS_AGE_KEY_FILE/
AGE_KEY_FILE) viavalidate_secret_path()and rejects paths starting with-
(flag-injection guard). Unit testsage_identity_file_traversal_rejectedand
age_identity_file_flag_injection_rejectedcover both rejection cases (SEC-3)..github/SECURITY.md: security policy with contact address, response SLA, supported
version matrix, and a summary of implemented controls (L-5).docs/security.md: new sections covering consumer SQL injection protection,
CNPG TLS configuration, andageidentity file security (SEC-1, SEC-2, SEC-3).
Changed
aqueduct plan --fail-on-drift: drift count now includessource_deltasand
consumer_deltasin addition todeltas, matching the already-correctstatus
implementation. Previously, consumer-layer and source-layer drift was silently
ignored (M-2).aqueduct destroy: changed fromconnect()toconnect_and_migrate()so a stale
catalog schema is auto-migrated before destroy queries it (M-5).release.yml: removed globalFORCE_JAVASCRIPT_ACTIONS_TO_NODE24env var — all
actions in the workflow use modern Node.js runtimes that do not require this
workaround (M-6). Addedjust gen-docsstep to regeneratedocs/api-reference.md
on every release tag.ingest.rstest helpers:fs::write(...).unwrap()replaced with descriptive
.expect("...")calls so a failing CI write produces a clear error message (M-7).fmt.rs: renamedKNOWN_FRONTMATTER_KEYStoKNOWN_DIRECTIVE_KEYSwith updated
doc comment explaining it is used to skip already-emitted keys, not enforce ordering
(L-1).main.rs: replaced stale// L7 (v0.14)tracking comment with descriptive
per-platform CI env var documentation (L-3).Cargo.toml: replaced stale# DEP-1 (v0.14): pre-declare serde_yamlcomment with
# YAML serialization for plan/status/diff --format yaml output.(L-4).- Six user-facing integration files updated to ref...
v0.18.0
This release closes every open security finding from the Phase 4 engineering assessment, bringing pg_aqueduct to zero open CVEs and zero open critical or high security issues. The headline change is that consumer view bodies are now validated as a single SELECT statement before any database call is made — meaning a misconfigured or maliciously crafted migration file containing DDL like CREATE TABLE or DROP TABLE can never reach the database, even if it somehow made it into your migrations directory. This is an important defence-in-depth layer for teams running Aqueduct in automated CI/CD pipelines.
Two additional critical security fixes ship in this version: the CloudNativePG preview client now validates TLS certificates by default (previously it silently accepted any certificate), and the age secret backend now validates the identity file path for both path traversal and flag injection before passing it to the age subprocess. Together with the consumer SQL validation, these three fixes mean pg_aqueduct meets the full OWASP A03 (Injection) and A02 (Cryptographic Failures) controls relevant to its operation.
Beyond security, this version improves day-to-day operational quality: aqueduct plan --fail-on-drift now correctly detects drift in consumer views and source tables (not just stream tables), aqueduct destroy auto-migrates a stale catalog before querying it, and several documentation files and CI template files that had stale version pins are updated to match the current release. A new .github/SECURITY.md establishes the project's vulnerability reporting policy.
Added
validate_consumer_sql_is_single_select()is now called immediately before every
CREATE OR REPLACE VIEWin the executor (ManageConsumerViewandSwapConsumerViews
arms). A migration file containing injected DDL is rejected with
AqueductError::UntrustedSqlBodybefore any database call is made (SEC-1).- Integration test
consumer_sql_injection_rejected_at_applyconfirms that a consumer
migration with aCREATE TABLEbody is rejected at the executor level (SEC-1). - CNPG preview HTTP client now validates TLS certificates by default. Set
CNPG_INSECURE_SKIP_VERIFYto skip verification in development environments only.
Emits awarn!when the env var is set (SEC-2). - Unit test
cnpg_tls_default_does_not_skip_verificationconfirms the env-var gate
logic (SEC-2). agesecret backend now validates the identity file path (fromSOPS_AGE_KEY_FILE/
AGE_KEY_FILE) viavalidate_secret_path()and rejects paths starting with-
(flag-injection guard). Unit testsage_identity_file_traversal_rejectedand
age_identity_file_flag_injection_rejectedcover both rejection cases (SEC-3)..github/SECURITY.md: security policy with contact address, response SLA, supported
version matrix, and a summary of implemented controls (L-5).docs/security.md: new sections covering consumer SQL injection protection,
CNPG TLS configuration, andageidentity file security (SEC-1, SEC-2, SEC-3).
Changed
aqueduct plan --fail-on-drift: drift count now includessource_deltasand
consumer_deltasin addition todeltas, matching the already-correctstatus
implementation. Previously, consumer-layer and source-layer drift was silently
ignored (M-2).aqueduct destroy: changed fromconnect()toconnect_and_migrate()so a stale
catalog schema is auto-migrated before destroy queries it (M-5).release.yml: removed globalFORCE_JAVASCRIPT_ACTIONS_TO_NODE24env var — all
actions in the workflow use modern Node.js runtimes that do not require this
workaround (M-6). Addedjust gen-docsstep to regeneratedocs/api-reference.md
on every release tag.ingest.rstest helpers:fs::write(...).unwrap()replaced with descriptive
.expect("...")calls so a failing CI write produces a clear error message (M-7).fmt.rs: renamedKNOWN_FRONTMATTER_KEYStoKNOWN_DIRECTIVE_KEYSwith updated
doc comment explaining it is used to skip already-emitted keys, not enforce ordering
(L-1).main.rs: replaced stale// L7 (v0.14)tracking comment with descriptive
per-platform CI env var documentation (L-3).Cargo.toml: replaced stale# DEP-1 (v0.14): pre-declare serde_yamlcomment with
# YAML serialization for plan/status/diff --format yaml output.(L-4).- Six user-facing integration files updated to reference version
0.18.0:
.github/workflows/aqueduct-plan.yml,.github/workflows/aqueduct-apply.yml,
ci/gitlab/aqueduct.gitlab-ci.yml(also fixes archive name pattern to match
the release pipeline:aqueduct-{version}-{os}-{arch}.tar.gz),
.pre-commit-hooks.yaml,docs/api-reference.md,docs/introduction.md(CI-1). version-lintCI job extended to check nine files:README.md,
docs/installation.md,docs/api-reference.md,docs/introduction.md,
ci/gitlab/aqueduct.gitlab-ci.yml,.pre-commit-hooks.yaml,
.github/workflows/aqueduct-plan.yml,.github/workflows/aqueduct-apply.yml,
andROADMAP.md. Fails if any disagrees with the workspaceCargo.tomlversion
(CI-1, L-6).ROADMAP.mdstatus line updated tov0.18.0 released(L-6).- Workspace version bumped to
0.18.0.
v0.17.0
Added
--patroni-endpoint <url>flag onaqueduct apply. Between each migration step,
the CLI callsGET <endpoint>/master; a non-200 response marks the migration
interruptedand aborts (DOC-2 / v0.17-A).aqueduct_core::hamodule:HaBackendenum,detect_ha_backend(),
check_still_primary()(usespg_is_in_recovery()), andcheck_patroni_primary()
HTTP check (DOC-2 / v0.17-A).- Between-step primary re-check via
SELECT pg_is_in_recovery()in the executor.
On failover, migration is markedinterrupted(v0.17-A). - Per-step JSON events emitted to stderr:
step_start,step_complete,step_failed
withstep_index,step_type,migration_id,project, andduration_msfields
(v0.17-B). Documented indocs/cli-events-schema.json. aqueduct auditsubcommand: lists recent migrations with step counts, error messages,
and durations. Supports--format table|json|yamland--limit N(v0.17-B).aqueduct.migration_historySQL view (catalog v8): joinsaqueduct.migrationsand
aqueduct.migration_stepsfor per-step audit queries (v0.17-B).aqueduct_core::catalog::LIST_MIGRATIONS_FOR_AUDIT_SQLquery constant (v0.17-B).- Prometheus metrics endpoint behind
--features metricsonaqueduct-cli.
Pass--metrics-addr 0.0.0.0:9090to expose/metricsduringapply(v0.17-B). - Catalog schema v8: adds
'interrupted'to themigrations.statuscheck constraint
and a partial index forinterruptedrows. Includes auto-migration from v7 (v0.17-A). - GitHub-native build provenance attestation in release workflow via
actions/attest-build-provenance@v2. Verify withgh attestation verify(v0.17-C). release-verifyCI job: downloads the linux-amd64 release archive, verifies its
SHA256 checksum, and confirmsaqueduct --versionmatches the tag (v0.17-C).- Version linting CI step (
version-lintjob): fails if README.md status banner,
docs/installation.md, andCargo.tomlworkspace version disagree (v0.17-D). just publish-dry-runrecipe for dry-run crates.io publish verification (v0.17-C).
Changed
- Catalog schema bumped from v7 to v8 (
CATALOG_SCHEMA_VERSION = 8). aqueduct-coreandaqueduct-testkitCargo.toml:publish = true;
aqueduct-cli:publish = false(v0.17-C).docs/ha-operations.mdrewritten to accurately describe implemented behaviors:
check_still_primary(), Patroni/mastercheck, per-step events,aqueduct audit,
and the recovery runbook. Speculativesystem_identifier/timeline_idcontent removed
(v0.17-D).docs/installation.md: updated version example to0.17.0, MSRV to 1.88,
added build provenance verification section (v0.17-C/D).ci.ymlsecurity-audit job: removed--ignore RUSTSEC-2025-0052(httpmock replaced).httpmockreplaced withwiremockinaqueduct-coredev-dependencies; all five
HTTP mock tests updated to the async wiremock API (v0.17-D).
v0.16.0
Added
OutputModeenum (Human,Json,Yaml,Quiet,Porcelain) and
OutputEmitterstruct inaqueduct-cli::output; global--quietand
--porcelainflags are now routed through the emitter (ERG-1, M11).serde_yamlworkspace dependency; YAML output inplan,status, anddiff
subcommands now usesserde_yaml::to_stringinstead of hand-rolled format strings,
correctly escaping quotes, colons, and newlines (ERG-3).just gen-docsrecipe that regeneratesdocs/api-reference.mdfrom
aqueduct --helpoutput; stale references to--patroni-endpoint,--timeout,
--lock-timeout, and--no-costremoved (ERG-4, M3).- Binary CLI tests (
assert_cmd) for every subcommand's exit codes and--help
output:quiet_suppresses_decorative_output,porcelain_outputs_key_value_only,
yaml_escapes_quotes_and_newlines,plan_help_documents_fail_if_changed,
apply_help_documents_dry_run,status_help_documents_fail_on_drift,
diff_help_documents_fail_on_drift,destroy_help_documents_dry_run,
rollback_help_documents_dry_run,plan_missing_dsn_exits_2,
validate_exits_0_on_valid_files,validate_differential_ivm_unsupportable_fails,
validate_format_json_produces_valid_json,version_flag_outputs_semver(TEST-2). --poll-intervalflag onaqueduct status(PERF-2).- Spec-hash caching in
status --watch: migration files are only reloaded when any
.sqlmtime changes; live state always polled fresh (PERF-2). postgres_version_matrix_min_supportedintegration test verifying the full
create/apply/status cycle on PostgreSQL 18+ (H10).CONTRIBUTING.mddocumenting Docker/Testcontainers prerequisites,justrecipes,
integration test environment variables, PG version matrix, coding standards, and
a step-by-step guide for adding cookbook recipes.
Changed
fmt::format_migrationnow returnsResult<Option<String>>and propagates IO
errors;aqueduct fmtprints a diagnostic when a file cannot be read instead of
silently treating it as empty (L2).CANONICAL_KEY_ORDERconstant inaqueduct-core::fmtrenamed to
KNOWN_FRONTMATTER_KEYSto reflect its actual role (L3).TestDb::connection_stringfield visibility narrowed frompubtopub(crate)
to prevent external crates from depending on the internal representation (L14).docs/security.md: AWS, GCP, and Vault secret backend rows updated from
"Planned (v0.12)" to "Implemented"; injected SQL trust-boundary note updated to
describe theCatalogSchemanewtype introduced in v0.14.docs/api-reference.mdregenerated fromaqueduct --helpoutput.
Fixed
aqueduct status --format yamlnow produces valid YAML that passes
serde_yaml::from_strround-trip including special characters.aqueduct plan --format yamlcorrectly escapes project names containing
quotes, colons, and newlines.
v0.15.0
Added
- Saga-style compensating steps:
CreateStreamTableandDropStreamTablewrite a
compensating action toaqueduct.ddl_logbefore executing DDL, enabling crash
recovery via--resume. --force-retry <step-index>and--force-skip <step-index>flags on
aqueduct applyto advance past ambiguous steps (require--yes).CatalogSchemanewtype validating catalog schema names at parse time, rejecting SQL
injection markers, non-identifier characters, leading digits, and reserved system
schema names (pg_catalog,information_schema, allpg_*prefixed names).ExecutionContextenum withApply,Rollback,Promote, andDryRunvariants
replacing the optional builder pattern; named constructorsfor_apply(),
for_rollback(), andfor_promote().StartBlueGreenDeploymentplan step inserting a row into
aqueduct.blue_green_deploymentswithstatus = 'active'as the first post-lock
step.- All
SwapConsumerViewssteps wrapped in a singleBEGIN ... COMMITblock for an
all-or-nothing consumer view swap. - Real RLS policy capture via
pg_policies/pg_class.relrowsecuritybefore
DropStreamTable;RecreatePolicystep restores policies after table recreation;
failures are non-fatal and logged toaqueduct.ddl_log. rebuild-may-affect-rlslint warning when aFULL-mode table has RLS enabled.- Batch convergence polling:
WaitForConvergenceissues a single
SELECT ... WHERE table_name = ANY($2)per poll interval instead of one query per
node. convergence_poll_interval_ms(default 500 ms) andconvergence_timeout_secs
(default 300 s) options inBuildPlanOptions.validate_migration_files_diagnosticandvalidate_dag_diagnosticpublic functions
returning structuredDiagnosticSetentries with file paths, error codes, and
severity levels.aqueduct validate --format jsonoutput matching thelintJSON schema.- Catalog schema v7:
aqueduct.ddl_loggainsmigration_id bigintand
compensating_sql text;aqueduct.blue_green_deploymentsgains
rolled_back_at timestamptzand an updatedbg_status_checkconstraint accepting
'rolled_back'.
Changed
- Executor writes
status = 'swapped'after the atomic consumer view swap and
status = 'retired'afterRetireBlueSchema. - Integration tests now target
postgres:18-alpineexclusively (PostgreSQL 18+
required by pg_trickle).
v0.14.0
Status: Released
All roadmap items for v0.14 "Phase 17 — Failure Safety, CI/CD Trustworthiness &
Security Hardening" are implemented and tested.
Highlights
-
Correctness fixes (CORR-1 through CORR-8): Seven correctness bugs were found
and fixed. Progress is now preserved on recoverable failures (CORR-1); the heartbeat
correctly signals the main executor loop via atokio::sync::watchchannel when the
advisory lock is lost (CORR-3); promotion filters the destination state by project
(CORR-4); the promote command migrates the catalog before use and records spec_jsonb
(CORR-5); consumer view drops now delete the catalog row so drift counts stay
accurate (CORR-7); and status drift counting now sums stream, source, and consumer
deltas (CORR-8). -
Catalog v6: The catalog schema is bumped to version 6. The new migration adds
thepgtrickle_mock.scheduler_statetable used by integration tests to assert that
pause_scheduler/resume_schedulercalls are routed correctly (TEST-3). -
Security hardening (SEC-1 – SEC-3):
- SEC-1: Keyword-value DSNs (e.g.
host=... password=secret) are now also
checked for plaintext passwords; previously only URI-style DSNs were validated. - SEC-2: Consumer SQL bodies are validated to be a single
SELECTstatement
before the view is created, preventing DDL injection via migration files. - SEC-3: The read-only helper now issues
BEGIN READ ONLYbefore setting
SET LOCAL statement_timeout, so the timeout is correctly scoped to the
transaction. Thesanitize_statement_timeoutallowlist prevents injection via
the timeout argument.
- SEC-1: Keyword-value DSNs (e.g.
-
Ergonomics (ERG-2):
aqueduct destroynow exits with code 2 on error (via
anyhow::bail!) instead of callingstd::process::exit(1)directly, ensuring the
error message is routed through the standard error handler. -
Lint fixes (L1, L7):
- L1: The DSN-redaction regex in the CLI is now compiled once via
LazyLock
instead of being recompiled on every call. - L7: CI environment variables (
GITHUB_ACTIONS,CI,GITLAB_CI,
CIRCLECI) are now checked for the string value"true"rather than merely being
set, preventing false positive CI detection.
- L1: The DSN-redaction regex in the CLI is now compiled once via
-
CI/CD fixes (CI-1 – CI-3):
- CI-1: The composite
planandapplyGitHub Actions now map the runner
OS/architecture to the correct release archive suffix (linux-amd64,
linux-arm64,macos-arm64,macos-amd64,windows-amd64). - CI-2: The action-smoke workflow now invokes the composite actions end-to-end
using a locally-packaged archive (via the newlocal-archiveinput). The
migration file path was corrected tomigrations/streams/event_count.sql. - CI-3:
mdbook testis now a blocking step (removedcontinue-on-error: true).
The docs-lint job no longer checks for a non-existentexportsubcommand, and
the subcommand reference check now fails the job if any documented subcommand is
missing from the binary.
- CI-1: The composite
-
Test infrastructure (TEST-3): The
pgtrickle_mockschema now includes a
scheduler_statetable. The mockpause_scheduler/resume_schedulerfunctions
insert and delete rows in this table, enabling integration tests to assert scheduler
state transitions. Nine new integration tests cover all CORR, SEC, and TEST items. -
pg_versionin JSON output (M15):aqueduct status --format jsonnow includes
a"pg_version"field.
Migration notes
-
Catalog: Run
aqueduct init(oraqueduct apply) to apply the v5→v6 catalog
migration. The migration is fully backward-compatible. -
Consumer SQL validation: If you have consumer migration files whose SQL body is
not a pureSELECTstatement,aqueduct applywill now reject them with error
code 1307. Update those files to use a singleSELECT.
v0.13.0
Status: Released
All roadmap items for v0.13 "Phase 16 — Blue/Green End-to-End, IMMEDIATE Mode &
Advanced Features" are implemented and tested.
Highlights
-
Blue/green migrations (
--strategy blue-green): The planner now generates the
full blue/green step sequence —CreateGreenSchema,CreateStreamTableInGreen,
WaitForConvergence(with real pg_trickle polling),SwapConsumerViews, and
RetireBlueSchema— for topology-restructuring diffs. -
IMMEDIATE refresh mode:
RefreshMode::Immediateis parsed from
-- @aqueduct:refresh_mode = "IMMEDIATE", classified, stored, and correctly handled
in rebuild paths viaPauseImmediate/ResumeImmediatesteps. The new
--no-immediate-downgradeflag rejects plans that would temporarily degrade an
IMMEDIATE table. -
Pre/post migration hooks: SQL statements from
[apply.hooks] preandpost
inaqueduct.tomlare emitted asRunHooksteps. The pre-hook fires immediately
afterLockDag; the post-hook fires just beforeRecordSnapshot. -
Immutable plan artifacts (
--out/--plan):aqueduct plan --out plan.json
writes the plan (with a SHA-256 spec hash) to a file.aqueduct apply --plan plan.json
loads and executes the pre-computed plan, validating the spec hash to reject stale
artifacts. -
Step observability (
migration_steps): The catalog is bumped to v5, adding
aqueduct.migration_steps. Every executed step writes arunningrow at start and
adone(orfailed) row at completion, enabling audit and replay. -
Import baseline snapshot:
aqueduct importnow callsensure_catalog_current
and records version 1 inaqueduct.dag_versions. A subsequentaqueduct plan
returns an empty plan. -
Real preview backends: The
create_preview_cnpgandcreate_preview_neonstubs
are replaced with real HTTP implementations using the CNPG Kubernetes API and the
Neon management API respectively. -
Rollback classification (
--within-window/--accept-data-loss): The rollback
command now classifies each change asSafe,PointInTime, orDataLossbased on
when the version was applied relative to the pg_trickle WAL retention window. -
JSON schema versioning (
schema_version: 1): All structured JSON events emitted
by the CLI now include"schema_version": 1. The schema is published at
docs/cli-events-schema.json.
v0.11.0
Status: Released
All roadmap items for v0.11 "Phase 14 — Diagnostic Quality, CLI Surface &
Documentation Correctness" are complete. This release focuses on making the CLI
output trustworthy, the error messages actionable, and the documentation honest.
What's new
Diagnostic Infrastructure
-
Q-08: New unified
Diagnostictype (aqueduct_core::diagnostic) replaces
the separateValidationError,LintDiagnostic, and ad-hoc parse error strings.
DiagnosticSetcollects diagnostics with file/line/column context. Both the
validate and lint subsystems now emitDiagnosticSetresults. -
Q-04: Stable numeric error codes (
error_code()method onAqueductError).
Error categories: 1000–1099 database, 1100–1199 configuration, 1200–1299 planning,
1300–1399 execution, 1400–1499 catalog. Codes are stable across releases. -
Q-03:
ManageWalSlotplan step uses typedWalSlotActionenum (Create/Drop)
instead of a free-formString. Eliminates the "Unknown action" dead code path.
CLI Surface Correctness
-
U-02: Fixed
aqueduct planexit semantics. Previously exited 1 for any non-empty
plan. Now exits 0 by default; exits 1 only when--fail-if-changedor
--fail-on-driftis explicitly set. CI pipelines that just want to see a plan
without failing now work correctly. -
U-03: New
--fail-on-driftflag foraqueduct diff. Exits 1 when any delta is
non-Unchanged, exits 0 for a clean diff. -
U-05:
executor.execute()now returnsExecutionResult { migration_id, dag_version }
instead of a bareu64. Theapply_completeJSON event on stderr now emits both
migration_id(aqueduct.migrations row id) anddag_version(bigserial) as separate
fields. -
U-06/SEC-01: Password redaction for all connection strings displayed in error
messages.redact_dsn()replaces passwords with***in both URL-form and
keyword-value DSNs. -
U-10: YAML output format for
aqueduct status(use--format yaml).
Read-Only Safety (D-03/SEC-08)
aqueduct planandaqueduct statusnow useconnect_read_only(), which issues
SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLYimmediately after connecting.
These commands cannot accidentally mutate data.
Documentation Correctness
-
D-01: README updated to v0.11.0 status and feature list.
-
D-04/D-05: Blue/green migration strategy (still planned for v0.13) is now clearly
marked as[PLANNED]indocs/api-reference.md,docs/ha-operations.md, and the
apply action YAML. The--strategy blue-greenflag in the apply action is disabled
(no-op) pending the v0.13 implementation. -
D-07: Unimplemented features (YAML output, IMMEDIATE mode, real secret backends)
moved from[Released]to[Planned]sections in CHANGELOG. -
D-09/SEC-02:
docs/security.mdnow clearly marks AWS/GCP/Vault secret backends
as "Planned (v0.12)". Onlyenv,sops, andageare implemented today. -
D-11:
docs/ha-operations.mdmaintenance window documentation updated to remove
blue/green from the implementedmaintenance_window_applies_tovalues. -
D-12: Example workflow version pins in
aqueduct-plan.ymlandaqueduct-apply.yml
updated from@v0.4.0to@v0.11.0.
Code Quality
-
Q-01: Step registry contract test — exhaustive match on all
PlanStepvariants
ensures every variant has a non-emptydescription(). Adding a new variant without
updatingdescription()is now a compile error. -
Q-02: Typed
PlanExecutorconstructors:for_apply(),for_rollback(),
for_promote(),for_dry_run(). The genericnew()constructor is still available. -
Q-06: Typed DAG state wrappers:
DesiredDagState,LiveDagState,
RecordedDagStatenewtypes wrappingDagState. Prevents mixing state kinds in
function signatures. -
Q-09:
lib.rsexportsCLI_VERSION = env!("CARGO_PKG_VERSION")as the single
source of version truth. All commands useenv!("CARGO_PKG_VERSION")directly.
CI Improvements
-
M-13:
docs-buildjob in CI runsmdbook buildandmdbook teston every push. -
U-01/D-02:
docs-lintjob in CI builds the release binary and verifies that
every subcommand responds to--help, and that the binary--versionmatches
Cargo.toml. -
New
docs-clirecipe injustfilegenerates a Markdown CLI reference from
--helpoutput.
Security
- SEC-07: New
docs/roles.sqlcontains SQL grant templates for the four aqueduct
roles:aqueduct_planner,aqueduct_applier,aqueduct_preview,aqueduct_destroy.
v0.10.0
Status: Released
All roadmap items for v0.10 "Phase 13 — Safety Contract Repair & Multi-Project
Isolation" are complete. This release repairs the safety contract (lock lifecycle,
resume, heartbeat, scheduler pause ordering) and establishes the multi-project
isolation invariant required for shared PostgreSQL deployments.
What's new
Critical Correctness Fixes
-
C-01 / C-11: Consumer-only plans are no longer silently skipped as no-ops.
PlanSummarynow tracksconsumer_creates,consumer_drops, andconsumer_alters
separately;is_empty()andtotal_changes()include consumer deltas. All renderer
totals updated. -
C-03:
RecordSnapshotnow usesquery_onewithRETURNING versionto capture
the actual bigserial value assigned by the database instead of recording the planned
version number. -
C-04: Backfill steps are documented as "Triggered by pg_trickle on table
creation — no explicit wait" in renderers and log messages. -
C-05:
read_live_state()now populatessourcesby deserialising the latest
spec_jsonbfromaqueduct.dag_versionsinstead of always returningvec![]. -
C-06 / C-07:
read_live_state()andread_live_consumers()both accept an
optionalprojectparameter and filter results via the new
aqueduct.stream_table_ownershiptable.destroy_projectverifies ownership before
dropping any table; use--force-unownedto bypass. -
C-09:
check_pgtrickle_version()probesto_regprocedure(...)before calling
the function to avoid a database error when pg_trickle is absent. -
C-10:
classify_delta()no longer panics on missingdesired/actualin the
AlterQueryarm; returnsRebuildinstead. -
C-12:
read_live_state()project filter enablesdiff --from last-appliedand
diff --from desiredmodes.
Safety & Idempotency Fixes
-
S-01 / S-02: Failed migrations are now marked
recoverable_failureand can be
resumed. On resume,LockDagalways re-executes — DDL steps are only skipped for
steps already confirmed complete in the checkpoint. -
S-03: Scheduler pause now happens inside the
LockDagexecutor arm, after the
lock is acquired, preventing a pre-lock pause from affecting other projects. -
S-04: Heartbeat SQL is now holder-bound
(WHERE project = $1 AND holder = $2). Zero rows affected is detected and logged as
a lock-stolen warning. -
S-05: The project lock is released on all exit paths (success and error) via an
explicit cleanup block after the step loop. -
S-06: Step progress checkpoint writes are now fatal — a failed write returns an
error rather than silently continuing. -
S-07:
aqueduct apply --dry-runuses a plain connection (no catalog migration)
to avoid upgrading the schema during previews. -
S-10:
DROP TABLEfallback indestroyand the executor no longer uses
CASCADE. Dependent objects must be dropped manually; use--force-cascadeto
opt in. -
S-11:
aqueduct rollbacknow enforces the--accept-data-lossflag when the
rollback plan contains rebuild-class steps. -
S-12: First-time stream table creations increment
create_countand not
rebuild_count, preventing false-positive maintenance-window blocking on initial
deployments. -
S-13: Lock holder string is now
aqueduct/{version}/{hostname}/{pid}/{ts}
making concurrent processes distinguishable. -
S-14: Scheduler pause failure is now a fatal error by default.
Catalog Schema v3
- Added
aqueduct.stream_table_ownership(project, schema_name, table_name, managed_since)
table with aPRIMARY KEY (schema_name, table_name)constraint. status_checkconstraint onaqueduct.migrationsnow includesrecoverable_failure.- Auto-migration from v2 to v3 applied on
connect_and_migrate.
New Tests (6)
test_consumer_only_apply_end_to_end: consumer view create with no stream table changes.test_concurrent_apply_race: two applies, second getsLockContention.test_stale_plan_detection: plan from_version mismatch vs DB version.test_two_project_isolation: destroy project A, project B tables intact.test_resume_skips_non_safety_steps: LockDag always at index 0.test_heartbeat_lock_is_holder_bound: wrong holder returns 0 rows.