-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ele 1684 create an unlinked tables macro #523
Changes from 3 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,59 @@ | ||||||
{% macro get_tables_without_models(deprecated_models_path=none) %} | ||||||
{% set model_schemas, model_tables = elementary.get_model_schemas_and_tables(deprecated_models_path) %} | ||||||
{% set db_tables = elementary.get_db_tables_in_schemas(model_schemas) %} | ||||||
{% for db_table in db_tables %} | ||||||
{% if db_table not in model_tables %} | ||||||
{% do print(db_table) %} | ||||||
{% endif %} | ||||||
{% endfor %} | ||||||
{% endmacro %} | ||||||
|
||||||
|
||||||
{% macro get_db_tables_in_schemas(schemas) %} | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What do you think about calling it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the thought was to separate |
||||||
{% set tables = [] %} | ||||||
{% for schema in schemas %} | ||||||
{% set db_name, schema_name = schema.split('.') %} | ||||||
{% set schema_relation = api.Relation.create(db_name, schema_name).without_identifier() %} | ||||||
{% set relations = dbt.list_relations_without_caching(schema_relation) %} | ||||||
{# list_relations_without_caching can return either a list of Relation objects or an agate depending on the adapter #} | ||||||
{% if relations.append is defined %} | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not do There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is pretty hacky, but there isnt an |
||||||
{# relations is a list of Relation objects #} | ||||||
{% for relation in relations %} | ||||||
{% do tables.append(schema ~ "." ~ relation.identifier) %} | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I usually prefer using |
||||||
{% endfor %} | ||||||
{% else %} | ||||||
{# relations is an agate #} | ||||||
{% for relation in elementary.agate_to_dicts(relations) %} | ||||||
{% do tables.append(schema ~ "." ~ relation.name) %} | ||||||
{% endfor %} | ||||||
{% endif %} | ||||||
{% endfor %} | ||||||
{% do return(tables) %} | ||||||
{% endmacro %} | ||||||
|
||||||
|
||||||
{% macro get_model_schemas_and_tables(deprecated_models_path=none) %} | ||||||
{% set model_schemas = [] %} | ||||||
{% set model_tables = [] %} | ||||||
|
||||||
{% set relevant_nodes = [] %} | ||||||
{% for model_node in graph.nodes.values() | selectattr('resource_type', '==', 'model') %} | ||||||
{% if not deprecated_models_path %} | ||||||
{% do relevant_nodes.append(model_node) %} | ||||||
{% endif %} | ||||||
{% if deprecated_models_path and not model_node.original_file_path.startswith(deprecated_models_path) %} | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
{% do relevant_nodes.append(model_node) %} | ||||||
{% endif %} | ||||||
{% endfor %} | ||||||
{% for model_node in relevant_nodes %} | ||||||
{% set model_schema = model_node.database ~ "." ~ model_node.schema %} | ||||||
{% set model_table = model_schema ~ '.' ~ model_node.name %} | ||||||
{% if model_schema not in model_schemas %} | ||||||
{% do model_schemas.append(model_schema) %} | ||||||
{% endif %} | ||||||
{% if model_table not in model_tables %} | ||||||
{% do model_tables.append(model_table) %} | ||||||
{% endif %} | ||||||
{% endfor %} | ||||||
{% do return([model_schemas, model_tables]) %} | ||||||
{% endmacro %} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From this macro you could infer that it's somewhat related to sourced, snapshots, seeds, etc.
Maybe call it something in the area of
leftover
ordangling
?Not super critical I suppose.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i like dangling, on it