You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The dbt-redshift adapter plugin inherits from the dbt-postgres adapter plugin. This reflects the fact that they share a lot of code: connection mechanism (psycopg2), basic syntax, types, materializations, etc. (This can be obfuscated by the fact that their shared implementations of macros are also often the default__ dbt implementations.)
Today, when dbt dispatches package_name.macro_name macro on Redshift, it looks for:
There are cases today where Redshift uses the default__ implementation and Postgres uses a custom postgres__ implementation, e.g. dbt_utils.dateadd.
I don't think this is a breaking change per se, because Redshift largely supports PostgreSQL syntax, but we may want to add implementations that explicitly preserve existing behavior:
This will enable maintainers of similar-yet-slightly-different plugins, e.g. dbt-sqlserver and dbt-synapse, who can then "share" macro implementations between parent and child adapters. See microsoft/dbt-synapse#18.
It's possible to imagine many more adapter inheritances in the future:
a dedicated dbt-databricks or dbt-delta that inherits from dbt-spark
a dbt-hive that inherits from dbt-spark or dbt-presto
a dbt-materialize that inherits from dbt-postgres
The text was updated successfully, but these errors were encountered:
friendly, curious bump. is this still something in consideration for Margaret Mead. Relatedly, I just peeped MaterializeInc/materialize-dbt-utils. Pretty cool to see folks catching dispatch fever!
Yes, this is still something I'd like to do for Margaret Mead! The change itself is very straightforward. We'll want to add some tests for it, and it will likely require a few changes in packages, especially dbt-utils.
Describe the feature
The
dbt-redshift
adapter plugin inherits from thedbt-postgres
adapter plugin. This reflects the fact that they share a lot of code: connection mechanism (psycopg2
), basic syntax, types, materializations, etc. (This can be obfuscated by the fact that their shared implementations of macros are also often thedefault__
dbt implementations.)Today, when dbt dispatches
package_name.macro_name
macro on Redshift, it looks for:my_project.redshift__macro_name
my_project.redshift__macro_name
package_name.postgres__concat
package_name.postgres__concat
I think it should instead look for:
my_project.redshift__macro_name
my_project.postgres__macro_name
my_project.default__macro_name
package_name.redshift__macro_name
package_name.postgres__macro_name
package_name.default__macro_name
This is a possibility we weighed in #2679, to the point that we make it possible as a one-line change:
https://github.com/fishtown-analytics/dbt/blob/5ba5271da99bc1ef7fbad2f7d0b45634087300fa/core/dbt/context/providers.py#L104-L109
What's the risk?
There are cases today where Redshift uses the
default__
implementation and Postgres uses a custompostgres__
implementation, e.g.dbt_utils.dateadd
.I don't think this is a breaking change per se, because Redshift largely supports PostgreSQL syntax, but we may want to add implementations that explicitly preserve existing behavior:
Who will this benefit?
This will enable maintainers of similar-yet-slightly-different plugins, e.g.
dbt-sqlserver
anddbt-synapse
, who can then "share" macro implementations between parent and child adapters. See microsoft/dbt-synapse#18.It's possible to imagine many more adapter inheritances in the future:
dbt-databricks
ordbt-delta
that inherits fromdbt-spark
dbt-hive
that inherits fromdbt-spark
ordbt-presto
dbt-materialize
that inherits fromdbt-postgres
The text was updated successfully, but these errors were encountered: