Skip to content

v1.12.3

Latest

Choose a tag to compare

@github-actions github-actions released this 17 Nov 19:11

Before you upgrade

  • generate airflow-dags now requires the Airflow version (#682). DAGs generated for Airflow 2 don't load on Airflow 3 and the other way around, so the command stops without generating anything unless you pass --airflow-version 2|3 or set it in .dbt_coves/config.yml:

    generate:
      airflow_dags:
        airflow_version: 3 # or 2, matching your Airflow

    Datacoves users: the VS Code task that generates a DAG from the open YML file runs in your dbt project folder. Set airflow_version in that folder's .dbt_coves/config.yml (the one under $DATACOVES__DBT_HOME, not the repository root), or the task fails after the upgrade. If one repository serves both Airflow 2 and Airflow 3 environments, read the value from an environment variable you set per environment: airflow_version: "{{ env_var('MY_AIRFLOW_VERSION') }}".

  • blue-green stops before deploying if production has grants it can't carry across the swap (#677). If the production database or one of its schemas is granted to a database role, share, user or application, the deploy now stops before dbt build with a list of those grants, where it used to swap and silently drop them. Production isn't touched.

  • Copied generate sources templates need updating (#681, and the model_name fix below). If you copied the templates with dbt-coves generate templates, use the new source and model_name variables, or copy the templates again. See Source names in templates.

blue-green

  • Database parameters (e.g. DATA_RETENTION_TIME_IN_DAYS), the database comment, MANAGED ACCESS on schemas, and future and inherited grants on the database and its schemas now survive the swap. Before, a deploy silently reset them, so objects built during the deploy could end up without their reader grants (#677).
  • The Snowflake connection is closed when the run ends (#683).
  • New docs section on views: a view that names its database breaks after the swap, and how to avoid it with a ref() override.

generate sources

  • The staging model's source() call now matches sources.yml. With the default templates, generated Snowflake staging models used to fail dbt parse (#681).
  • Snowflake tables and schemas created with quoted names (lowercase, mixed case, spaces, reserved words) work: they keep their exact case and get a quoting: block in sources.yml. JSON flattening quotes the names too (#681).
  • --flatten-json-fields yes no longer skips tables without JSON columns (#685).
  • Flattened JSON keys no longer produce duplicate column names. A key whose name is already taken, e.g. a type key in two JSON columns plus a TYPE column, is prefixed with its JSON column's name (properties_type). Other keys keep their names.
  • The staging model's properties file names the model after its .sql file, so a models_destination like stg_{{relation}}.sql no longer produces properties dbt can't match ("Did not find matching node for patch").

generate properties

  • --profiles-dir, --profile, --target, --vars and --target-path are passed to the dbt ls run that lists your models. Properties used to be generated from the default target's tables (#680).

generate airflow-dags

  • Generated DAGs are adapted to the Airflow version: decorator imports, renamed arguments (e.g. schedule_interval → schedule for Airflow 3) and warnings for behaviour changes (#682).
  • --validate-operators no longer crashes on valid operators (#682).
  • The dbt generators read manifest.json from the project's target path instead of always target/ (#685).
  • A --yml-path that doesn't exist is a clear error. A single YML file outside the Datacoves DAGs YML folder no longer fails with ValueError: ... is not in the subpath of ....

generate metadata

  • JSON columns are sampled with quoted names on Snowflake, so quoted tables and columns work (#685).

All commands

  • --vars works again with dbt 1.8 and later. Any value crashed with TypeError: 'str' object is not a mapping.
  • Errors are reported as a message, without a Python traceback, unless they're unexpected; add -vv or --log-level debug to see it. Errors are no longer printed twice (#685).
  • Properties files dbt-coves rewrites no longer re-wrap long descriptions or leave trailing spaces.

Dependencies

  • Allows snowflake-connector-python 5 and python-slugify 9 (#683).
  • Security updates to locked dependencies, including gitpython and oauthlib (#679).