-
Notifications
You must be signed in to change notification settings - Fork 68
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Include DBT DRY RUN as a utilities plugin (#1164)
Co-authored-by: Pat Nadolny <patnadolny@gmail.com>
- Loading branch information
1 parent
ea01635
commit da5f240
Showing
3 changed files
with
58 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -599,3 +599,4 @@ utilities: | |
postgres: postgres | ||
sqlfluff: sqlfluff | ||
superset: meltano | ||
dbt-dry-run: autotraderuk |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
definition: | | ||
dbt-dry-run is a meltano plugin based on the awesome tool created by the team at AutoTraderUk. | ||
It allows dbt-bigquery users to validate if their models will run (without encuring costs). | ||
It leverages bigquery's [dry-run](https://cloud.google.com/bigquery/docs/dry-run-queries) feature. | ||
**A note** | ||
This plugin is at it's most powerful (in our opinion) in CI (we use github actions). It allows us to ensure that our changes will not break production. | ||
domain_url: https://github.com/autotraderuk/dbt-dry-run | ||
executable: dbt-dry-run | ||
keywords: | ||
- dbt | ||
- bigquery | ||
- testing | ||
- continuous-integrations | ||
- meltano_edk | ||
label: dbt-dry-run (BigQuery Only) | ||
logo_url: /assets/logos/utilities/dbt-dry-run.png | ||
maintenance_status: active | ||
name: dbt-dry-run | ||
namespace: dbt-dry-run | ||
next_steps: | | ||
To run this plugin for the first time, simply specify you DBT_BIGQUERY_PROFILES_DIR, DBT_BIGQUERY_PROJECT_DIR and DBT_DRY_RUN_TARGET. | ||
You will also need to compile your dbt models using the command `dbt-bigquery compile`. | ||
To run your plugin, you will need to define a target as well. We propose to easily to this by creating this custom command: | ||
``` | ||
commands: | ||
run: | ||
args: | ||
--target ${DBT_DRY_RUN_TARGET} | ||
--verbose | ||
definition: | | ||
Allows you to run the plugin. Requires that you specify a profile target. | ||
Note - Requires you run `dbt-bigquery compile` before executing. | ||
executable: dbt_dry_run_invoker | ||
``` which will allow you to run the plugin at the specified target profile. | ||
Once this is done, simply run `dbt-dry-run:run` and watch the magic happen! | ||
pip_url: git+https://github.com/potloc/dbt-dry-run-ext.git | ||
ext_repo: https://github.com/potloc/dbt-dry-run-ext | ||
repo: https://github.com/autotraderuk/dbt-dry-run | ||
settings: | ||
- name: project-dir | ||
value: ${MELTANO_PROJECT_ROOT}/transform/ | ||
env: DBT_PROJECT_DIR | ||
description: Location of the directory hosting your `dbt_project.yml` file. | ||
kind: string | ||
- name: profiles-dir | ||
kind: string | ||
value: ${MELTANO_PROJECT_ROOT}/transform/profiles/bigquery/ | ||
env: DBT_PROFILES_DIR | ||
description: Location of the directory hosting your `profiles.yml` file. | ||
- name: target | ||
kind: string | ||
description: Target profile in DBT. | ||
variant: potloc |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.