Feature: Support cross-project Group evaluation #60
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description and Motivation
Within dbt-core, it is possible to assign models and other resources to
groups
. These groups can be used to restrict other downstream resources fromref
-ing the resource if they are not part of the same group. Due to the way that dbt-core instantiatesModelNodes
based onModelNodeArgs
, it has not been possible to enforce group reference constraints across project boundaries, andref
-ing protected nodes from across projects has historically yielded error messages lacking the group name of the upstream node (#59).This pull request unblocks cross-project group evaluation by extending our internal
LoomModelNodeArgs
construct to include agroup
string, and wraps the existingModelNode.from_args()
classmethod with an internalmodel_node_wrapper
, which inject the upstream node'sgroup
string into the resultingModelNode
. I also added a similar wrapper to ensure that downstream projects have their upstream-projects injected as valid group names during group validity checks. The metadata will NOT be passed into the downstream manifest, but this is a good first step.Along the way, I found another defect in dbt-core's evaluation of access evaluation for private models, which prevents grouped private models in projects with restricted access from using generic tests (dbt-labs/dbt-core#10134). This means that we cannot enforce checks against
protected
models as described in #43. I'll restore this functionality once the underlying defect has been resolved in dbt-core.