ETLantic 0.37.0

[0.37.0] - 2026-07-30

Adopter summary

Who must act	Anyone pinning etlantic==0.36.x or plugins with <0.37
Breaking	Dependency floor becomes etlantic>=0.37.0,<0.38 (stable-foundation line). Remaining demoted root aliases and DataContractModel removed
Upgrade	pip install 'etlantic==0.37.0' and matching plugins / medallantic==0.37.0. See Migration 0.36 → 0.37
Rollback	Re-pin 0.36.0 minors together; re-validate
Security	Compatibility fixtures and reports remain secret-free; production allowlist pins fail closed; security verification matrix ships with automated checks

Added

• Stable-foundation acceptance suite evidence for items 1–21
  (tests/stable_foundation/ + scripts/check_stable_foundation.py)
• Security verification matrix with owner + automated check per control
  (docs/02_FOUNDATIONS/SECURITY_VERIFICATION_MATRIX.md,
  scripts/check_security_matrix.py)
• Diagnostic-code stability tiers inventory
  (docs/10_REFERENCE/DIAGNOSTIC_STABILITY_TIERS.md,
  scripts/check_diagnostic_stability.py)
• Release baseline / burn-in fixtures for v0_37 under
  tests/fixtures/releases/ and tests/fixtures/burn_in/
• What's New / Migration / Exit Gate / Findings documentation for 0.37

Changed

• Official package versions align at 0.37.0; plugins require
  etlantic>=0.37.0,<0.38 (external echo reference pin
  etlantic>=0.37,<0.38)
• Supported security release line is 0.37.x
• etlantic.testing graduates from preview freeze to stable
  application-pipeline testing foundation
• Public surface / protocol freeze inventories align to the 0.37 reference
  envelope; first-party packages declare etlantic>=0.37.0,<0.38
• etlantic.scheduler/1 remains stable MVP; etlantic.quality/1 remains
  provisional; DataFusion remains experimental; Arrow stays Gate A only
• PyPI Beta classifier retained on core and first-party packages

Removed

• Remaining demoted root facade aliases (_DEMOTED_ALIASES empty;
  _REMOVED_0_37 raises with migration guidance)
• DataContractModel root / contracts compatibility alias; use
  etlantic.Data / ContractModel

Fixed

• Root removal regressions covered by
  tests/unit/test_root_removals_0_37.py and testing-foundation suites
• Docs / release-facts / artifact verification contract pointed at v0.37.0
  (expected after tag; digests filled at publish)

Upgrade notes

• Pin core and matching plugins / medallantic together at ==0.37.0
• Replace demoted root imports and any DataContractModel usage before
  upgrade
• Plugin authors: raise the floor to etlantic>=0.37.0,<0.38 and re-run
  public conformance
• Prefer graduated public etlantic.testing for application pipeline cases

ETLantic 0.36.0

[0.36.0] - 2026-07-30

Adopter summary

Who must act	Anyone pinning etlantic==0.35.x or plugins with <0.36
Breaking	Dependency floor becomes etlantic>=0.36.0,<0.37 (no wire-schema reset). Bare run-report metadata keys migrate to namespaced keys. Production allowlists require non-empty version pins. Soft-continued runs report PARTIAL. Missing implementations fail at plan (PMPLAN301). Reliability partition-check / quality-trends require evidence flags
Upgrade	pip install 'etlantic==0.36.0' and matching plugins / medallantic==0.36.0. See Migration 0.35 → 0.36
Rollback	Re-pin 0.35.0 minors together; re-validate
Security	Compatibility fixtures and reports remain secret-free; production allowlist pins + outbound default-deny fail closed; schema history / authoring / Safe I/O hardened; digests/attestations as shipped

Added

• Joint compatibility burn-in evidence for 0.34 → 0.35 and 0.35 → 0.36
  (current-reader and isolated-wheel reader/writer paths)
• Release baseline manifests and known-defect fixtures under
  tests/fixtures/releases/
• Burn-in goldens for v0_34–v0_36 alongside historical v0_24–v0_27
• Application-pipeline testing burn-in against the frozen preview minimum
  contract (etlantic.testing; foundation graduation remains 0.38)
• What's New / Migration / Exit Gate / Findings documentation for 0.36

Changed

• Official package versions align at 0.36.0; plugins require
  etlantic>=0.36.0,<0.37
• etlantic.scheduler/1 promoted to stable MVP (Prefect bounds) on the
  foundation path
• etlantic.quality/1 remains provisional (outside the full stable-foundation
  claim)
• Run-report bare metadata keys migrate to namespaced keys without silent
  field loss
• Production plugin_allowlist entries require non-empty version pins;
  package names are canonicalized for matching
• Reliability CLI: partition-check requires --observed; quality-trends
  requires --values (preview-only)

Fixed

• Forward compatibility for 0.35.0 bare-key run-report metadata via documented
  migration and known-defect fixture
  (tests/fixtures/releases/v0_35/known_defects/run_report_bare_metadata.json)
• SecretValue is no longer a dataclass (blocks dataclasses.asdict plaintext
  leaks)
• Schema history rejects list-of-lists, nested cells, and JSON-string row
  payloads
• Authoring serialization rejects DSN keys and URL userinfo in definition
  values
• Safe I/O: empty approved_roots fail closed; symlink_policy=reject denies
  any symlink component; invalid profile safe_io no longer silently dropped;
  lock/append files use 0o600
• Production outbound requires default_deny and non-empty allowed_hosts
• Profile warns when name looks like production without
  security_mode=production
• Planner fails closed (PMPLAN301) without a real implementation; plan graphs
  are frozen after build
• Contract validation requires schema fingerprint match for shared published
  ODCS ids; mixed typed/untyped ports fail
  closed
• FailureAction.CONTINUE soft-skips report PARTIAL (never pure SUCCEEDED)
• Spark sinks no longer coerce Delta for overwrite_partition / bare
  pyspark provider
• LazyFrame record counts report unknown (None) instead of a false zero
• OPS examples, FastAPI OpenAPI version, and Medallantic ROADMAP release header
  aligned to 0.36.0
• Package READMEs, optional-API metadata, planning status, and Beta maturity
  language aligned to 0.36; release-artifact docs no longer predict an SBOM
  outcome before the tag workflow runs
• Documentation checks now reject stale current_minor / current-boundary
  markers, contradictory package maturity text, and premature release-asset
  claims
• Log redaction covers auth/jwt/bearer keys and JWT-shaped tokens

ETLantic 0.35.0

[0.35.0] - 2026-07-30

Adopter summary

Who must act	Anyone pinning etlantic==0.34.x or plugins with <0.35
Breaking	Dependency floor becomes etlantic>=0.35.0,<0.36 (no wire-schema reset)
Upgrade	pip install 'etlantic==0.35.0' and matching plugins / medallantic==0.35.0
Rollback	Re-pin 0.34.0 minors together; re-validate
Security	Migration analysis remains secret-free; digests/attestations as shipped

Added

• Public authoring helpers: inspect_definition, rewrite_definition,
  definition_provenance
• Application-pipeline testing preview in etlantic.testing
• Medallantic M7 SparkForge project inventory and safe native generation

Changed

• Package versions and plugin pins advance to the 0.35.x line

ETLantic 0.34.0

[0.34.0] - 2026-07-30

Security and reliability hardening

• Phase-aware PMPLUG402: authorize/selected-engine denials block
  ensure_plugins / run / compile / doctor / execute (discovery siblings stay
  non-blocking); run maps all raised PMPLUG* to TRUST_FAILURE; doctor
  plugin trust failures exit 11
• Manual overlays cannot spoof builtin local/env allowlist exemption;
  secret / observability / run-history / event-consumer / storage registers are
  production-allowlist gated; authorize path no longer exempts engine-name spoofs
• Profile constructor rejects plaintext secrets; plan/profile serialization
  refuses non-SecretRef values
• Profile assets / binding deserialize reject URL userinfo so credential URLs
  cannot enter plan snapshots; report nested metadata bags and extension keys
  (including dsn / connection_string / jdbc_url and URL values) fail
  closed; CLI emit_payload redacts before echo
• Delta merge_keys validated as safe identifiers; non-Delta MERGE/UPSERT
  fails closed (no overwrite fallback); Delta capabilities require
  delta.tables.DeltaTable
• SQL executor diagnostics redact exception text; PostgreSQL MERGE without
  update columns fails closed unless explicit insert-ignore intent
• Airflow compile preserves schedule timezone (ScheduleIntent.from_mapping
  no longer invents UTC), stubs on any ERROR diagnostic (not only schedule),
  refuses unsupported schedules with a non-schedulable stub, and secret-scans
  rendered source (including URL userinfo)
• Trusted-SQL profile mismatch uses PMEXEC353 and is CONTINUE/SKIP hard-fail;
  redact_message redacts full URL userinfo
• CSV/JSON sinks honor write_mode and optional SafeIoPolicy; memory
  merge/upsert fails closed; FileStateStore.commit reads previous inside
  locked RMW; durable_audit event consumers fail closed
• Medallantic: missing validation/artifact status fail closed; run status
  reconciled when steps failed; empty SparkForge write mode rejected
• FileStateStore fails closed on corrupt state; schema history verifies
  stored fingerprints; file schema/run-history providers default
  fail_closed=True

Added

• Lifecycle event correlation (etlantic.lifecycle_event/1) with plan/region/backend fields
• Observability provider protocol (etlantic.observability/1) and runtime bridge
• Run history provider protocol with file and in-memory reference implementations
• Event consumer protocol with reference trend consumer
• Profile keys: observability_providers, run_history_provider, event_consumers, observability_delivery
• Conformance suites: observability, run history, event consumer, production profile
• Medallantic M6: explain_medallion_plan, lifecycle views, medallion profile templates
• CLI etlantic report query and workspace file run history (.etlantic/history/)
• What's New / Migration / Exit Gate 0.34 documentation

Changed

• Official package versions align at 0.34.0; plugins require etlantic>=0.34.0,<0.35
• Terminal run reports may include metadata.observability event/delivery summary
• etlantic reliability quality-trends uses event consumer summary when --values omitted

Fixed

• Selected-engine PMPLUG402 now maps validate/plan CLI exit to TRUST_FAILURE (sibling discovery 402s remain non-blocking)
• Plugin registry descriptors retain package identity (distribution_name / package) after authorize-before-load
• Manual register_*_plugin refuses unauthorized overlays when a production profile is active
• Missing selected engines under production emit blocking PMPLUG404 trust diagnostics
• Plan snapshots with name=production require security_mode=production
• PySpark provider/plugin advertise Delta/merge capabilities only when delta-spark is importable; declared cache/checkpoint failures fail closed
• Medallantic missing run/step statuses fail closed; live IR scrubbing covers non-metadata secret keys; transform_ref imports support deny-by-default allowlists
• FileStateStore uses locked read-modify-write; schema/run history can fail closed on corrupt durable files
• Schema-drift / trust failures cannot be CONTINUE/SKIP'd; plan allow_trusted_sql is intersected with the live profile
• Polars/Pandas advertise only append/overwrite write extras; Airflow fails closed on non-cron/manual schedules; keyring capability ads match sync lookup; mounted-file secret errors omit absolute paths
• Structured run logs can fan out to observability providers via runtime bridge
• durable_audit fail-closed on run-history event append (not only terminal reports)
• File run-history JSONL append uses locked O_APPEND (cross-process safe; no full-file read)
• Run history query/list: orphan reports, idempotent create_run, UTC filter coercion, sorted report queries
• Production conformance requires run_history_provider when observability_delivery is durable_audit
• etlantic report query supports --since / --until; documentation drift for 0.34 M6 surfaces
• SQL temp-table materialization validates column identifiers (injection fail-closed)
• Asset URLs with userinfo credentials rejected from plan bindings (secret-free plans)
• Builtin allowlist exemption no longer trusts third-party engine/name spoofs
• Mounted-file secrets confined under mount root (path traversal fail-closed)
• Schema history scans field metadata, denies list-of-row bags, and uses collision-resistant subject paths
• Manual register_*_plugin overlays gated by production plugin_allowlist
• Non-blocking PMPLUG402 ignored consistently in trust exit, validation discovery, run/compile/doctor, and transform dispatch
• Quality membership fails closed on null; uniqueness skips null keys and rejected rows
• PipelinePlan.from_dict(verify=True) requires a non-empty fingerprint; production snapshot mode/name consistency checked
• Airflow artifact secret scan covers every strategy; plan snapshots keep only referenced secrets
• Airflow reference operator fails closed unless ETLANTIC_AIRFLOW_REFERENCE_STUB=1
• etlantic run maps validation/trust failures to documented exit codes; compile uses trust exit
• Nested pipeline definitions upgrade/validate to etlantic.pipeline/1
• Contract-diff toolkit exceptions fail closed as BREAKING drift
• --quiet still emits JSON/SARIF on trust gate failures
• Outbound emit payloads redacted in reports; authoring/redaction vocab includes pwd / AWS access keys
• Step TimeoutError is not retried under the default retry policy
• Spark cancel surfaces failures; PySpark advertises Delta only when delta is importable
• Strict production extension metadata rejects nested secret-like keys

ETLantic 0.33.0

[0.33.0] - 2026-07-29

Added

• SQL dialect Tier A/B matrix in etlantic-sql (SQLite + PostgreSQL live; others gated)
• PostgreSQL sql_merge via INSERT … ON CONFLICT; model DDL + primary-key validation
• Lazy SQL transform claim + planner sql_fusion / transaction-scope evidence
• Async SQLAlchemy execution path when the URL uses an async driver
• Medallantic live from_sql_pipeline_builder bridge (PMSQ350+)
• Moltres / SQLAlchemy-native rules (quality.moltres_expr, MDL132)
• etlantic.testing.run_sql_builder_differential_suite with classified fixtures
• What's New / Migration / Exit Gate 0.33 documentation

Changed

• Official package versions align at 0.33.0; plugins require etlantic>=0.33.0,<0.34
• SqlTransformCompiler advertises lazy=True alongside eager=True

Fixed

• Authorize plugins on package/distribution identity only (engine short names can no longer spoof allowlists); emit PMPLUG403 for invalid pins in the authorize path; exempt builtins in lifecycle policy
• Validation/ensure_plugins no longer fail closed on sibling PMPLUG402 denials for non-selected engines; CLI trust exit ignores non-blocking 402s
• PlanningContext rediscovers and clears non-builtin plugins when the profile allowlist key changes; empty production discovery clears the registry
• Spark cancel treats capability-probe failures as not advertised (fail closed)
• Preserve env builtin registry stubs across profile switches; etlantic profile validate maps all PMPLUG* errors to trust exit
• Portable SQL transform compiler enables merge compile for PostgreSQL dialects; Cockroach-via-postgresql:// URLs classify as Tier B without merge
• SQL hello PyPI companion reuses the plugin engine (examples/sql_hello_pypi.py); production sample banner/links/docs gates hardened
• SQL merge capability tests are dialect-aware; dialect-tier suite skips cleanly without SQLAlchemy so non-SQL CI jobs collect

Upgrade notes

• Pin core and official plugins to ==0.33.0 (matching minors while pre-1.0).
• Read Migration 0.32 → 0.33
  and What's new in 0.33.
• SQL merge requires PostgreSQL (sql_merge); SQLite remains fail-closed.
• Prefer medallantic.migrate.sql.from_sql_pipeline_builder for live SQL builders.
• Moltres-only rules fail closed with MDL132 until an evaluator exists.
• Out-of-monorepo plugins: bump core pins to etlantic>=0.33.0,<0.34
  (see etlantic-plugin-echo 0.33.0).

ETLantic 0.32.0

[0.32.0] - 2026-07-28

Added

• storage.delta.* capability extras and plan-time fail-closed PMPLAN440 / PMPLAN441
• Catalog mutation policy (PMCAT100) and JDBC/asset binding refs
• Spark protocol: cancel, storage ops, cache/checkpoint points, logical-step identity
• etlantic-pyspark Delta optimize/vacuum/history/time-travel/schema-evolution paths
• Medallantic live from_pipeline_builder bridge and Column-rule path (MDL130)
• Real PySpark df→df callable transforms; fluent builder rules= on silver/gold
• Runtime map implementation_overrides + invalidation
• etlantic.testing.run_sparkforge_differential_suite with classified fixtures
• What's New / Migration / Exit Gate 0.32 documentation

Changed

• Official package versions align at 0.32.0; plugins require etlantic>=0.32.0,<0.33
• Medallantic Delta capability map uses fine-grained storage.delta.* extras
• etlantic-pyspark claims dataframe=True (lazy implication honesty)

Fixed

• Spark region compile/execute records cache points and logical identities

Upgrade notes

• Pin core and official plugins to ==0.32.0 (matching minors while pre-1.0).
• Read Migration 0.31 → 0.32
  and What's new in 0.32.
• Local / Polars / Pandas-only adopters can skip most M4 Spark/Delta notes;
  re-pin plugins and continue.
• SparkForge / Delta users: prefer
  medallantic.migrate.sparkforge.from_pipeline_builder for live builders;
  expect storage.delta.* capability negotiation (PMPLAN440 / PMPLAN441).
• Non-portable Column rules need quality.pyspark_column or fail closed
  (MDL130).
• Out-of-monorepo plugins: bump core pins to etlantic>=0.32.0,<0.33
  (see etlantic-plugin-echo 0.32.0).

ETLantic 0.31.0

[0.31.0] - 2026-07-28

Added

• IncrementalStrategy and StateStore (MemoryStateStore, FileStateStore) with commit-after-materialization
• WriteMode.SKIP_IF_EXISTS and write capability extras (write.*) with PMPLAN430 / PMPLAN431
• Run-intent write-mode resolution (VALIDATE / INITIALIZE / INCREMENTAL / REFRESH / STANDARD)
• Lifecycle policy helpers and Medallantic bronze/silver/gold facade defaults
• Medallantic live transform_ref execution (medallantic.callables) composed with quality gates
• enforce_accept_rates / MDL120 for threshold failures
• etlantic.testing.run_lifecycle_conformance_suite
• What's New / Migration / Exit Gate 0.31 documentation

Changed

• Official package versions align at 0.31.0; plugins require etlantic>=0.31.0,<0.32
• Local memory storage honors append and skip_if_exists write modes
• Medallantic skip/ignore write modes map to SKIP_IF_EXISTS
• Orchestrator records state_transitions on run reports when state commits

Fixed

• VALIDATE / no-write runs no longer advance incremental state

ETLantic 0.30.0

[0.30.0] - 2026-07-28

Added

• Provisional etlantic.quality/1 portable quality-expression AST + JSON Schema
• etlantic.quality helpers: serialize/fingerprint, ContractModel mapping, analyze, evaluate, make_quality_gate
• Plan metadata for quality gates (cost, capabilities, fallback evidence, validation_boundary)
• Plan-time fail-closed diagnostics PMPLAN420 / PMPLAN421 for unsupported required quality capabilities
• Portable quality conformance suite: etlantic.testing.run_quality_conformance_suite
• Polars / Pandas advertise portable quality rule capabilities
• Medallantic rule DSL (parse_rules_shorthand) and real quality-gate lowering (replaces MDL110 passthrough)
• Medallantic evaluate_accept_rates for layer threshold comparison
• What's New / Migration / Exit Gate 0.30 documentation

Changed

• Official package versions align at 0.30.0; plugins require etlantic>=0.30.0,<0.31
• Local stub registry advertises portable quality capabilities for plan-time negotiation
• Medallantic bronze steps with rules insert {name}__ingest + gate nodes
• Surface inventory marks etlantic.quality/1 provisional and adds quality lazy namespace

Fixed

• Quality gate factory resolves ContractModel port types and invalid-role rejected outputs
• Name-only custom_contract rules fail closed at evaluate time (no silent pass)
• Quality capability asserts run before native implementation selection so SQL/PySpark profiles emit PMPLAN420/421 instead of PMPLAN302
• Plan metadata recomputes quality fingerprints so embedded hashes cannot drift
• Optional required=False quality rules skip rejection while recording soft diagnostics
• Regex / range / length DSL evaluation no longer crashes the gate on bad inputs
• field_constraints_dict projects nullable, compare ops, and composite unique_fields
• Medallantic quality gates attach {step}__rejected sink Loads (invalid→sink wiring allowed)
• PySpark stops advertising invalid_row_separation without dataframe
• Surface / agent / capability / diagnostics catalogs reflect etlantic.quality for 0.30

ETLantic 0.29.0

[0.29.0] - 2026-07-28

Added

• Medallantic M1 native authoring: MedallionPipeline, MedallionBuilder, Bronze / Silver / Gold
• Declarative medallion documents and stable MDL1xx construction/graph diagnostics
• medallantic.migrate.sparkforge migration namespace (shared lowering with native path)
• Facade conformance kit: etlantic.testing.run_facade_conformance_suite
• Definition extension-bag validation on codec write; namespaced facade extensions copied onto plans
• etlantic.extensions.facade_provenance helper
• What's New / Migration / Exit Gate 0.29 documentation

Changed

• Official package versions align at 0.29.0; plugins require etlantic>=0.29.0,<0.30
• Medallantic IR adapter reuses shared lower_document path
• Facade packages documents the shipped conformance kit
• Plugin lifecycle allowlists accept engine/entry short names consistently with trust filtering
• Discovery APIs expose last_diagnostics when trust blocks loads

Fixed

• Medallantic LoweringError / AdapterError codes now match the primary ERROR diagnostic
• Native MedallionDocument.from_dict preserves explicit 0.0 accept rates
• Unknown medallion layers fail closed (MDL107) instead of coercing to gold
• Validation policies are unique per pipeline name + schema (no same-schema clobber)
• Document metadata and step description/tags survive definition extensions
• SparkForge IR cycle detection handles "step.result" source references
• etlantic_sparkforge submodule imports (adapt / ir / compat / reports / runtime_map)

ETLantic 0.27.0

[0.27.0] - 2026-07-28

Added

• Triple-minor burn-in fixtures (v0_26/) proving 0.26→0.27 upgrade without wire-schema reset
• Extended CI gates for pipeline and sibling codec burn-in across v0_24, v0_25, and v0_26
• What's New / Migration / Exit Gate 0.27 documentation
• medallantic package (replaces etlantic-sparkforge) for SparkForge migration adapter

Changed

• Official package versions align at 0.27.0; plugins require etlantic>=0.27.0,<0.28
• Plugin SDK /1 freeze re-scoped to 0.28+ (external feedback blocker remains open)
• Wire schema ranges document the triple-minor window
• Rename etlantic-sparkforge to medallantic; optional extra is etlantic[medallantic]

Fixed

• Close Profile.__init__ metadata validation bypass for production profiles
• Validate plan graph, region, boundary, and artifact metadata on decode; apply
  production strict mode from profile_snapshot.security_mode
• Whitelist first-party core plan metadata keys to stop spurious bare-key warnings
• Map etlantic plan trust-phase failures to exit code 11 (TRUST_FAILURE)
• Correct plugin pin in Migration 0.24→0.25 guide
• Refresh active docs, issue template, and examples README to 0.27.0
• Harden check_removed_root_imports and extend check_docs stale-pin detection
• Update burn-in script docstrings for v0_26 coverage

Removed

• Second-wave root facade aliases: reliability declarations (12), schema_drift helpers (8), registry descriptors (6); import from owning modules per Migration 0.26→0.27
• etlantic-sparkforge distribution (superseded by medallantic)