Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ADAP-464] Materialized Views - Non-incremental support #672

Open
Fleid opened this issue Apr 19, 2023 · 2 comments · May be fixed by #1255
Open

[ADAP-464] Materialized Views - Non-incremental support #672

Fleid opened this issue Apr 19, 2023 · 2 comments · May be fixed by #1255
Assignees
@mikealfare
Labels
enhancement New feature or request materialized_views user docs [docs.getdbt.com] Needs better documentation

Comments

@Fleid
Copy link
Contributor

Fleid commented Apr 19, 2023 •
edited

As commented in the core MV thread, BigQuery is adding support for non-incremental MVs.

From our side, it means that we have an additional setting to support for MVs in BQ: allow_non_incremental_definition: true/false. This setting is covered by on_configuration_change via DROP/CREATE since it can't be changed via ALTER.

When set, it requires max_staleness to be set. I'd rather leave BQ errors out here for now - this is a preview feature and I'm not sure this won't change before GA. We should not check this.

BigQuery

BigQuery Materialized Views

  • Configurations:
    • enable_refresh : BOOLEAN, optional (database level default true) (see lifecycle for effect)
    • refresh_interval_minutes : integer, optional (database level default at 30), only valid for enable_refresh true or default
    • max_staleness : time before bypassing the MV and running the statement instead (in preview but I like it!)
    • allow_non_incremental_definition : BOOLEAN, optional (database default false)
    • From models: Partition / Cluster, TTL, Labels
  • Lifecycle : default except Refresh is bypassed if auto_refresh
    • on_configuration_change.apply : enable_refresh/refresh_interval_minutes/max_staleness via ALTER, rest via DROP/CREATE
@Fleid Fleid added the enhancement New feature or request label Apr 19, 2023
@Fleid Fleid added this to the 1.6.0 milestone Apr 19, 2023
@Fleid Fleid self-assigned this Apr 19, 2023
@github-actions github-actions bot changed the title Materialized Views - Non-incremental support [ADAP-464] Materialized Views - Non-incremental support Apr 19, 2023
@Fleid
Copy link
Contributor Author

Fleid commented Apr 19, 2023

@mikealfare FYI

@mikealfare mikealfare self-assigned this Apr 19, 2023
@dataders dataders mentioned this issue Jul 14, 2023
Closed
2 tasks
@bnaul
Copy link

bnaul commented Nov 8, 2023 •
edited

@Fleid @mikealfare this has been out there for a while now, I think it's basically impossible at this point that they would drastically change the functionality, as they have written it up and pulled it into official client libraries. Happy to try my hand at a PR for adding this if it would be welcome! Changing to an "ask for forgiveness instead of permission" approach #1011

@bnaul bnaul mentioned this issue Nov 8, 2023
Closed
4 tasks
@mikealfare mikealfare removed this from the 1.6.0 milestone Dec 13, 2023
@Fleid Fleid removed their assignment Feb 14, 2024
@mikealfare mikealfare added the user docs [docs.getdbt.com] Needs better documentation label Jun 11, 2024
@mikealfare mikealfare linked a pull request Jun 11, 2024 that will close this issue
Draft
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mikealfare mikealfare

Labels
enhancement New feature or request materialized_views user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Milestone
No milestone
4 participants
@bnaul @Fleid @mikealfare @dbeatty10