Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snowflake: Enable choosing a warehouse at runtime #1358

Closed
cmcarthur opened this issue Mar 21, 2019 · 1 comment · Fixed by #1788
Closed

Snowflake: Enable choosing a warehouse at runtime #1358

cmcarthur opened this issue Mar 21, 2019 · 1 comment · Fixed by #1788
Assignees

Comments

@cmcarthur
Copy link
Member

Feature description

There are cases where you want a set of models in your project to run on a different Snowflake warehouse so that it completes more quickly. Right now you can do that with hacks involving multiple profiles or environment variables across multiple runs, but it would be easier to use if this was more configurable.

My best guess at how to implement this is to add a warehouse model config that can be specified in dbt_project.yml or in a config block. I think this field should be rendered so that it's configurable in dev vs. prod. When set, it will override the Snowflake warehouse to use for that model.

I think that would be difficult because of how model configs work, and how connections are established. (We can switch the warehouse after the connection is established, so maybe it's not a problem?) But, if that's a problem, I could imagine setting this at the run level, too. So every model for a given run would execute on a custom warehouse.

@krishbox
Copy link

@cmcarthur, another approach to do this would be to use a pre-hook with a USE WAREHOUSE query_wh;

@drewbanin drewbanin added this to the Louisa May Alcott milestone Sep 17, 2019
@beckjake beckjake self-assigned this Sep 26, 2019
beckjake added a commit that referenced this issue Sep 27, 2019
…del-config

Add runtime per-model warehouse config on snowflake models (#1358)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants