-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
merge exclude columns for incremental models #5457
Conversation
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide. |
Thanks @dave-connors-3! I'm tagging this In order to merge this PR, we'll want a functional test for It would make sense for this new test to live alongside the existing tests for |
Our team would love to have this feature. What is the status of this PR? |
@nathaniel-may could this be a PR that you prioritize the team to get some eyes on and review please? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code here is straightforward and does not look risky. This does not implement adapter zone tests, however, so that should be done in a separate ticket/pr.
## What are you changing in this pull request and why? Adding a section for `merge_exclude_columns` to go with the similar config `merge_update_columns`. This is functionality that was added [here](dbt-labs/dbt-core#5457) Closes #2197 <!--- Describe your changes and why you're making them. If linked to an open issue or a pull request on dbt Core, then link to them here! To learn more about the writing conventions used in the dbt Labs docs, see the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md). --> ## Checklist <!-- Uncomment if you're publishing docs for a prerelease version of dbt (delete if not applicable): - [ ] Add versioning components, as described in [Versioning Docs](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/versioningdocs.md) - [ ] Add a note to the prerelease version [Migration Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/guides/migration/versions) --> - [x] Review the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) and [About versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) so my content adheres to these guidelines. - [x] Add a checklist item for anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."
resolves #5260
Description
This PR allows a user to specify a list of columns not to update during an incremental run using the merge strategy. This is achieved via a new macro
get_merge_update_columns
that will take into account user input from eithermerge_update_columns
or the newmerge_exclude_columns
config to return the correct set of columns to update.This will need to be dispatched to accomodate slightly different behavior in the
dbt-spark
repository.Checklist
changie new
to create a changelog entry