Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

DBT Build Task #184

Merged
merged 7 commits into from
Apr 22, 2024
Merged

DBT Build Task #184

merged 7 commits into from
Apr 22, 2024

Conversation

robfreedy
Copy link
Contributor

Closes #176

Executes the 'dbt build' command within a Prefect task, and optionally creates a Prefect artifact summarizing the dbt build results. This is a building block for a future prebuilt dbt_flow() but can be used as an individual task.

Example

dbt_build_task(
    project_dir="/Users/rob.freedy/Documents/prefect/internal-prefect/prefect-dbt-examples/prefect_dbt_bigquery"
)

Screenshots

image

Checklist

  • References any related issue by including "Closes #" or "Closes ".
    • If no issue exists and your change is not a small fix, please create an issue first.
  • Includes tests or only affects documentation.
  • Passes pre-commit checks.
    • Run pre-commit install && pre-commit run --all locally for formatting and linting.
  • Includes screenshots of documentation updates.
    • Run mkdocs serve view documentation locally.

@robfreedy robfreedy requested a review from a team as a code owner April 19, 2024 14:38
Copy link
Contributor

@zzstoatzz zzstoatzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good! just a couple comments

prefect_dbt/cli/tasks.py Outdated Show resolved Hide resolved
dbt_cli_profile: Optional[DbtCliProfile] = None,
dbt_client: Optional[dbtRunner] = None,
create_artifact: bool = True,
artifact_key: str = "dbt-build-task-summary",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to parameterize this for a given task so the user doesn't always overwrite the same key by default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going to leave this as the default for the dbt build command because dbt build runs all of the models/tests/snapshots/seeds/etc. We will have other pre-built tasks with their own artifacts for each of the individual node types that I will make sure have unique keys for the artifacts

prefect_dbt/cli/tasks.py Outdated Show resolved Hide resolved
prefect_dbt/cli/tasks.py Outdated Show resolved Hide resolved
prefect_dbt/cli/tasks.py Outdated Show resolved Hide resolved
prefect_dbt/cli/tasks.py Outdated Show resolved Hide resolved
@zzstoatzz
Copy link
Contributor

@robfreedy looks like you might also need pre-commit install

@robfreedy robfreedy merged commit 50ec86a into main Apr 22, 2024
6 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DBT improvements: dbt_build_task
2 participants