Add table rename at the end of each refresh

[nbenezri]

Some tables take a long time to refresh, like:

09:12:05  119 of 175 START table model cdp_stripe.stripe__invoice_line_items ............. [RUN]
09:15:08  119 of 175 OK created table model cdp_stripe.stripe__invoice_line_items ........ [SELECT in 183.45s]

During this time - the table is not available to query in prod.
One way to resolve this is to run in the end:

DROP TABLE IF EXISTS cdp_stripe.stripe__invoice_line_items_prod;
ALTER TALBE cdp_stripe.stripe__invoice_line_items rename to cdp_stripe.stripe__invoice_line_items_prod;

Is there such a way to do it when each table is ready?

[fivetran-reneeli]

Hi @nbenezri thanks for opening this issue!

Just to confirm I understand, are you saying that after the refreshed table gets created again, you're unable to query from it? But by renaming the table then you can? I'm having trouble understanding why it's unable to be queried in the first place. Or, can you query in dev, just not in prod?

[nbenezri]

When the refreshes going on it is unavailable. In the above example, during 09:12:05 - 09:15:08.

[fivetran-reneeli]

Hi @nbenezri sorry for the delay! I understand the issue you're experiencing, though this is the way dbt works as is it runs a create or replace statement. I think the issue may be better resolved looking at how your environment is set up. I would suggest working with your engineering team to make sure your dashboards are still available during table refreshes-- there should be some kind of best practices for when prod goes offline temporarily.

[nbenezri]

Yeah, I will rename the tables after each refresh. I thought there might be something simpler. *Thanks,* *Nir Ben Ezri* *DBA Team Lead* *DistroKid* *https://distrokid.com* <https://distrokid.com/>

…

On Thu, Jan 26, 2023 at 11:24 PM Renee Li ***@***.***> wrote: Hi @nbenezri <https://github.com/nbenezri> sorry for the delay! I understand the issue you're experiencing, though this is the way dbt works as is it runs a create or replace statement. I think the issue may be better resolved looking at how your environment is set up. I would suggest working with your engineering team to make sure your dashboards are still available during table refreshes-- there should be some kind of best practices for when prod goes offline temporarily. — Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATR4QSVYVLWF5KAAO2SJP7LWULTSXANCNFSM6AAAAAAUDZTMCM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[fivetran-reneeli]

Yeah, this is something more likely within dbt-core or your internal architecture! Will close this issue as this falls outside the realm of our packages