Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use enabled config to implement cross-database support #59

Closed
drewbanin opened this issue May 16, 2019 · 2 comments
Closed

Use enabled config to implement cross-database support #59

drewbanin opened this issue May 16, 2019 · 2 comments
Milestone

Comments

@drewbanin
Copy link
Collaborator

Right now, Snowplow models are encoded as macros in order to implement cross-database support. Instead, we should experiment with making models specifically for each support adapter. I think broadly, this falls into:

  • default (snowflake/redshift/postgres)
  • bigquery

The models directory might look like:

models:
 - common/
 - default/
 - bigquery/

Every model would need to have a config like:

{{ config(enabled=(var('adapter') == 'default')) }}
or
{{ config(enabled=(var('adapter') == 'bigquery')) }}

I don't think we can configure this at a folder level in dbt_project.yml, but it might be worth exploring if there's some clever way to implement that.

The results of this experimentation might result is us wanting to keep the adapter macro pattern, but I'm interested in exploring alternative ways of supporting cross-db model implementations in packages!

@jtcohen6 jtcohen6 added this to the 0.8.0 milestone May 16, 2019
@jtcohen6 jtcohen6 modified the milestones: 0.8.0, 0.9.0 Oct 21, 2019
@jtcohen6
Copy link
Contributor

Resolved in #62. Once 0.16.1 comes out, and dbt supports returning boolean YML values from Jinja macros, we can pursue an even-more seamless implementation in dbt_project.yml.

@drewbanin
Copy link
Collaborator Author

Just an FYI - boolean yaml values will likely be 0.17.0, but regardless this all sounds great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants