Skip to content

Destinatoin snowflake: Add config option to enable time travel - #35754

Merged
Edward Gao (edgao) merged 6 commits into
masterfrom
edgao/snowflake/configurable_retention_period
Apr 10, 2024
Merged

Destinatoin snowflake: Add config option to enable time travel#35754
Edward Gao (edgao) merged 6 commits into
masterfrom
edgao/snowflake/configurable_retention_period

Conversation

@edgao

Copy link
Copy Markdown
Contributor

closes #21879

... also sets the retention period on the final tables, where previously we weren't. I could see an argument that - since we've been using the default (1 day) since dv2 release, that we should actually default to 1 day everywhere? but either way, wanted to get the bones of the PR out for review + trigger CI.

@edgao
Edward Gao (edgao) requested a review from a team as a code owner March 1, 2024 16:46
@vercel

vercel Bot commented Mar 1, 2024

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 10, 2024 7:41pm

@evantahler

Copy link
Copy Markdown
Contributor

I would vote for 1-day retention on all tables as the default, but that's a soft prefrence.

@edgao

Copy link
Copy Markdown
Contributor Author

sold on 1 day by default. I don't particularly want to write a migration for alter table blah set data_retention_days = <new value> though - i.e. existing raw tables will continue to use 0-day retention. (mostly b/c it's just extra complexity to implement. We never automatically drop/recreate raw tables, so users are free to manually do that alter table if they want)

@edgao
Edward Gao (edgao) merged commit 4fc7f1a into master Apr 10, 2024
@edgao
Edward Gao (edgao) deleted the edgao/snowflake/configurable_retention_period branch April 10, 2024 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/destination/snowflake

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Destination Snowflake: allow configuration of data_retention_time_in_days so we can set values greater than 0

4 participants