Skip to content

DRE v0.2.0

Choose a tag to compare

@github-actions github-actions released this 03 Oct 15:11
· 19 commits to master since this release
496678a

Connections and sources: one report can read from several systems. Each query can run on its own connection (queries[].profile), dbt-style sources declare the tables a project reads (sources/*.yml, {{ source('shop', 'orders') }}, dre run -s source:shop.orders, dre ls --resource-type source, dre validate --live checks declared columns), and DRE finds each query's connection without connecting by rendering every query once in a parse pass. A workbook can hold DuckDB, Postgres and Databricks tabs side by side. One session per connection; queries still run one at a time, in YAML order.

Needs the duckdb, postgres and databricks plugins 1.1.0 or later (they report their identifier quote character). Skills 2.0.0 support this release.

Breaking changes (see https://getdre.com/docs/migrating-to-0.2/):

  • profiles.yml: sources: is now connections: (sources: still works this release, with a warning). A profile's own target: is ignored: every run has one target, from --target, DRE_TARGET, target: in dre_project.yml, else dev. A destination with no entry for the target is skipped and logged.
  • Templates: connection.* replaces target.<field>, run.profile and run.source_type; target has only name; profile(role='source') is role='connection'; destination.* is new. Each removed name is an error saying what to write instead.
  • Every profile: value may use Jinja (var(), env_var(), run.*, target.name).
  • The manifest is schema 2 (a sources section, project.target, each query's connection and depends_on); run_results.json records each result set's connection.
  • Schedules: schedule-needs-anchor, schedule-too-frequent and schedule-seconds are now errors, as 0.1.2 announced.
  • dre new --source <plugin> is dre new --type <plugin> (--source still works).

Install: curl -fsSL https://github.com/get-dre/dre/releases/latest/download/install.sh | sh, pip install dre-cli, or see the README. Plugins are released separately (<package>-v<version> releases) and installed by dre on demand.

What's Changed

  • Skills 1.1.0: write, preview and orchestrate schedules by @allenhori in #91
  • Harden secret redaction across source diagnostics by @nijanthan-dev in #79
  • Move CLI DuckDB seeding into a private test helper by @nijanthan-dev in #81
  • Per-query connections, dbt-style sources and one target per run (0.2) by @allenhori in #92
  • Release 0.2.0; skills 2.0.0 by @allenhori in #93

Full Changelog: v0.1.2...v0.2.0