Skip to content

[Feature] Iceberg support for view materialization #341

Description

@dataders

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt-adapters functionality, rather than a Big Idea better suited to a discussion

Describe the feature

with dbt-labs/dbt-snowflake#321, dbt-snowflake now supports Iceberg table format with table_format="iceberg" for table, incremental, dynamic, and snapshot materializations. Other adapters already support Iceberg table format, and will hopefully soon standardize on the same config.

However, there is now a spec for Iceberg views. While not yet supported in any of the adapters that we support (except for Vanilla Spark), I expect that we'll see them eventually.

Perhaps this means that table_format might also soon be an option we'd want to extend to the view materialization. Maybe it's view_format, but I think it makes more sense to be consistent in config naming so that table_format may be set for a group of tables & views that a user desires to write to an Iceberg catalog.

{{
  config(
    materialized="view",
    table_format="iceberg"
)}}

Describe alternatives you've considered

No response

Who will this benefit?

from @Fokko:

With Iceberg having view support now, I think dbt and Iceberg are going to be a great fit and is going to open up the analytics landscape where you can do the heavy lifting with one engine and do the final analytics with another
The version of views currently is still bound to the engine that makes them (it is just SQL that's being stored), but we're transitioning into a intermediate-representation like Substrait, but that's going to need a bit more time/work

Are you interested in contributing this feature?

No response

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions