duckrun 0.3.1
duckrun is a dbt adapter built on dbt-duckdb — run dbt build/run/test/docs with DuckDB as the engine and Delta Lake as the only state. The adapter is stateless (in-memory DuckDB by default); Delta tables are the single source of truth, surfaced back as delta_scan views so {{ this }} and is_incremental() resolve across processes. Works locally and against OneLake / ADLS / S3, with append / merge / delete+insert / microbatch incremental strategies and on_schema_change.
New here? This is the dbt-adapter line (0.3+). The pre-0.3 standalone
duckrunAPI lives on a legacy branch, pinned asduckrun<0.3. 0.3.x is not a drop-in upgrade for 0.2.x — adopt it as a new adapter.
What's new in 0.3.1
- Hardened the AEMO ingestion model against slow/flaky networks: retry the
read_textsource-listing fetches and the GitHub backfill with exponential backoff; retry per-file downloads on timeouts (not just HTTP errors); gentler concurrency and request throttling; DUID reference downloads fall back to the existing local copy on failure. - Archive log writes are now an upsert (
mergeon file identity) instead ofappend, so the log stays idempotent and safe under concurrent runs.
Install
pip install -U duckrun