Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use depends_on.macros as input to state:modified #3278

Closed
jtcohen6 opened this issue Apr 20, 2021 · 0 comments · Fixed by #3559
Closed

Use depends_on.macros as input to state:modified #3278

jtcohen6 opened this issue Apr 20, 2021 · 0 comments · Fixed by #3559
Labels
enhancement New feature or request state Stateful selection (state:modified, defer)
Milestone

Comments

@jtcohen6
Copy link
Contributor

During state comparison, dbt knows which macros have been modified, but all it does today is log a warning if it finds one:

https://github.com/fishtown-analytics/dbt/blob/f73202734c08b85da739818af2aff74dcbc8638a/core/dbt/graph/selector_methods.py#L435-L445

Why just raise a warning? Well, even though we've long had depends_on.macros for models, we didn't have it for macros—so we didn't have a reliable way to know if a model depends on a macro that depends on a (changed) macro.

For an unrelated purpose, #3272 added a full depends_on.macros for macros, too. We could use this (now reliable) depends_on.macros as an input to state:modified, to select models that call (directly or indirectly) a modified macro.

I think this would look like:

  • Find the set of modified macros (we have this today)
  • Recursively find the "extended" set of macros that depend on a modified macro
  • Find the set of all nodes that depend on any macro in the extended set

I think this could be a simpler way to get at some of the functionality that #2714 described, though the proposal there is much more complex.

@jtcohen6 jtcohen6 added enhancement New feature or request state Stateful selection (state:modified, defer) labels Apr 20, 2021
@jtcohen6 jtcohen6 added this to the Oh-Twenty-One milestone Jun 14, 2021
@jtcohen6 jtcohen6 self-assigned this Jul 12, 2021
@jtcohen6 jtcohen6 removed their assignment Nov 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request state Stateful selection (state:modified, defer)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant