Skip to content

🚀 elevata v0.6.0 — Warehouse-Native Execution & Deterministic Historization

Choose a tag to compare

View all tags
@ilona0925 ilona0925 released this 14 Dec 14:41
· 57 commits to main since this release
v0.6.0
6d6fd47

This release marks a major inflection point for elevata.

With v0.6.0, elevata transitions from a metadata-driven SQL generation framework into a warehouse-native execution engine capable of running, tracking, and historizing data loads directly in the target system.

✨ Highlights

🧠 Warehouse-Native Execution

  • Introduction of a fully functional Execution Engine (elevata_load --execute)
  • SQL is rendered and executed directly in the target warehouse
  • Consistent execution contract across DuckDB, PostgreSQL, and SQL Server
  • Guardrails prevent accidental cross-system execution

📊 Load Run Observability

  • New warehouse-level execution log: meta.load_run_log
  • Captures:
    • target dataset & schema
    • execution timestamps and duration
    • execution mode (full / incremental)
    • success or failure status
  • Enables auditing, monitoring, and downstream observability use cases

🧬 Full SCD Type 2 Historization

  • Complete, metadata-driven SCD Type 2 pipeline
  • Automatic management of:
    • version lifecycle (version_started_at, version_ended_at)
    • state tracking (active, changed, deleted)
    • load lineage via load_run_id
  • Deterministic change detection using row-level hashing
  • History tables are append-only and never truncated

🔁 Incremental MERGE Pipelines

  • Robust incremental processing for Rawcore datasets
  • Dialect-aware MERGE generation:
    • native MERGE where supported
    • UPDATE + INSERT fallback where required
  • Integrated delete detection and version closing

🏗️ Automatic Target Provisioning

  • Schemas and meta tables are created automatically when missing
  • Idempotent DDL generation ensures safe repeated execution
  • No manual warehouse setup required for supported targets

📐 Dialect Parity & Technical Robustness

  • Aligned execution semantics across all supported dialects
  • Dialect-specific handling for:
    • hashing functions
    • timestamp literals
    • UPDATE syntax differences
  • Technical columns use optimized, semantically correct data types per backend

📚 Documentation

The architectural documentation has been fully aligned with the implemented execution model:

  • Load SQL Architecture
  • Historization Architecture
  • Dialect System & Execution Semantics
  • Target Backend Requirements

All documentation now describes the existing system, not planned or future behavior.

🧪 Stability

  • Comprehensive test coverage across load generation, execution, and historization
  • Cross-dialect validation (DuckDB, PostgreSQL, SQL Server)
  • All existing tests green

🎯 Summary

v0.6.0 establishes elevata as a deterministic, metadata-driven execution engine that runs directly inside the warehouse.

It provides a solid foundation for upcoming capabilities such as ingestion, data quality enforcement, and orchestration — without compromising architectural clarity or control.

© 2025 elevata Labs

Full Changelog: v0.5.3...v0.6.0

Assets 2
Loading