dbt 0.18.1b2

An initial release of 0.18.1b2

v0.18.1b1

Merge remote-tracking branch 'origin/releases/0.18.1b1' into dev/0.18.1

dbt 0.18.0

dbt 0.18.0 - Marian Anderson (September 3, 2020)

Links:

• 📖 Migration guide + docs

Breaking changes

• Previously, dbt put macros from all installed plugins into the namespace. This version of dbt will not include adapter plugin macros unless they are from the currently-in-use adapter or one of its dependencies #2590
• adapter_macro is no longer a macro, instead it is a builtin context method. Any custom macros that intercepted it by going through context['dbt'] will need to instead access it via context['builtins'] (#2302, #2673)
• adapter_macro is now deprecated. Use adapter.dispatch instead.
• Data tests are now written as CTEs instead of subqueries. Adapter plugins for adapters that don't support CTEs may require modification. (#2712)

Features

• Made project-level warnings more apparent (#2545)
• Added a full_refresh config item that overrides the behavior of the --full-refresh flag (#1009, #2348)
• Added a "docs" field to macros, with a "show" subfield to allow for hiding macros from the documentation site (#2430)
• Added intersection syntax for model selector (#2167, #2417)
• Extends model selection syntax with at most n-th parent/children dbt run --models 3+m1+2 (#2052, #2485)
• Added support for renaming BigQuery relations (#2520, #2521)
• Added support for BigQuery authorized views (#1718, #2517)
• Added support for altering BigQuery column types (#2546, #2547)
• Include row counts and bytes processed in log output for all BigQuery statement types (#2526)
• Added option "--adapter" to dbt init to create a sample profiles.yml based on the chosen adapter (#2533, #2594)
• Added support for Snowflake query tags at the connection and model level (#1030, #2555)
• Added new node selector methods (config, test_type, test_name, package) (#2425, #2629)
• Added option to specify profile when connecting to Redshift via IAM (#2437, #2581)
• Add more helpful error message for misconfiguration in profiles.yml (#2569, #2627)
• Added support for setting policy tags for BigQuery columns (#2586, #2589)
• Added --defer and --state flags to dbt run, to defer to a previously generated manifest for unselected nodes in a run. (#2527, #2656)
• Add a BigQuery adapter macro to enable usage of CopyJobs (#2709)
• Support TTL for BigQuery tables(#2711)
• Add better retry support when using the BigQuery adapter (#2694, follow-up to #1963)
• Added a dispatch method to the context adapter and deprecated adapter_macro. (#2302, #2679)
• The built-in schema tests now use adapter.dispatch, so they can be overridden for adapter plugins (#2415, #2684)
• Add support for impersonating a service account using impersonate_service_account in the BigQuery profile configuration (#2677) (docs)
• Macros in the current project can override internal dbt macros that are called through execute_macros. (#2301, #2686)
• Add state:modified and state:new selectors (#2641, #2695)
• Add two new flags --use-colors and --no-use-colors to dbt run command to enable or disable log colorization from the command line (#2708)

Docs

• Add "Referenced By" and "Depends On" sections for each node (docs#106)
• Add Name, Description, Column, SQL, Tags filters to site search (docs#108)
• Add relevance criteria to site search (docs#113)
• Support new selector methods, intersection, and arbitrary parent/child depth in DAG selection syntax (docs#118)
• Revise anonymous event tracking: simpler URL fuzzing; differentiate between Cloud-hosted and non-Cloud docs (docs#121)
• Add project level overviews (docs#127)

Fixes

• Fixed an error in create_adapter_plugins.py script when -dependency arg not passed (#2507, #2508)
• Remove misleading "Opening a new connection" log message in set_connection_name. (#2511)
• Now all the BigQuery statement types return the number of bytes processed (#2526).
• Adapter plugins can once again override plugins defined in core (#2548, #2590)
• Added --selector argument and support for selectors.yml file to define selection mechanisms. (#2172, #2640)
• Compile assets as part of docs generate (#2072, #2623)
• Fix Redshift table size estimation; e.g. 44 GB tables are no longer reported as 44 KB. #2702
• Fix issue where jinja that only contained jinja comments wasn't rendered. (#2707, #2178)
• dbt now validates the require-dbt-version field before it validates the dbt_project.yml schema (#2638, #2726)

Under the hood

• Upgraded snowflake-connector-python dependency to 2.2.10 and enabled the SSO token cache (#2613, #2689, #2698)
• Add deprecation warnings to anonymous usage tracking (#2688, #2710)
• Data tests now behave like dbt CTEs (#2609, #2712)
• Adapter plugins can now override the CTE prefix by overriding their Relation attribute with a class that has a custom add_ephemeral_prefix implementation. (#2660, #2712)
• Added 3 more adapter methods that the new dbt-adapter-test suite can use for testing. (#2492, #2721)
• It is now an error to attempt installing dbt with a Python version less than 3.6. (resolves #2347)
• Check for Postgres relation names longer than 63 and throw exception. (#2197)

Contributors

Thank you all for your contributions...

dbt 0.18.0rc1

An initial release of 0.18.0rc1

dbt 0.18.0b2

An initial release of 0.18.0b2

dbt 0.17.2

dbt 0.17.2 (July 29, 2020)

This is a bugfix release with one breaking change, which may be relevant to users of community-supported plugins.

Breaking changes (for plugins)

• The release argument to adapter.execute_macro no longer has any effect. It will be removed in a future release of dbt, likely v0.18.0. (#2650)

Features

• Added environment variables for debug-level logging (#2633, #2635)

Fixes

• When using the --fast-fail option with adapters that don't support cancelling queries (e.g. BigQuery), dbt will include the original database error in its log messages (#2644, #2646)
• dbt clean does not require a profile (#2620, #2649)
• On Redshift, the catalog contains information for all schemas in a project, rather than just the default (#2653, #2654)
• On Snowflake, close all connections so keepalive thread will exit (#2645, #2650)

Docs

• Fix background appearance of markdown ``` code blocks (docs#114, docs#115)

Contributors:

• @joshpeng-quibi (#2646)

dbt 0.17.2rc1

This is a bugfix release

dbt 0.17.2b1

Expose snowflake debug logs

dbt 0.17.1

dbt 0.17.1 (July 20, 2020)

This is a bugfix release with one breaking change and one enhancement of an existing feature.

Breaking changes

• dbt native rendering is no longer enabled by default, as it was in v0.17.0. It requires the use of opt-in filters: as_native, as_bool, and as_number. (#2612, #2618)

Features

• On BigQuery, persist_docs persists descriptions for nested columns (#2549, #2550)

Fixes

• Hash names of local packages (#2600)
• Handle too-long file paths on Windows (#2591, #2558, #2566, #2603)
• Fix a bug where persist_docs ignored seeds on BigQuery (#2598, #2601)
• Fix a bug where dbt config-version: 2 rendered +pre-hook and +post-hook fields too eagerly (#2583, #2854)
• Fix a bug where dbt compile and dbt ls would create schemas if they didn't already exist (#2525, #2528)
• Fix a bug where dbt deps ignored the --project-dir flag (#2519, #2534)
• Fix a bug where parallel RPC requests confounded each others' arguments ([#2484, #2554])
• Fix a bug where dbt ignored config aliases in config() calls and dbt_project.yml (#2557, #2559, #2575)
• Fix unclickable nodes in the dbt Docs DAG viz (#101)
• Fix relation names with null databases (i.e. Spark) in dbt Docs site (#96)

Contributors:

• @bodschut (#2550)

dbt 0.17.1rc4

This is a bugfix release