View: Add StorageTableIdentifier and storage-table field to ViewVersion (MV spec, part 1) #15523
Open
osscm wants to merge 1 commit intoapache:mainfrom
Open
View: Add StorageTableIdentifier and storage-table field to ViewVersion (MV spec, part 1) #15523osscm wants to merge 1 commit intoapache:mainfrom
osscm wants to merge 1 commit intoapache:mainfrom
Conversation
…on (MV spec, part 1) Introduces foundational data model for Materialized View support per the MV spec (apache#11041). Adds StorageTableIdentifier, extends ViewVersion with an optional storage-table field, updates the parser, and adds tests. Plain views are fully backward compatible (storageTable() returns null).
Author
|
cc @anuragmantri |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First in a series of incremental PRs toward implementing the Iceberg Materialized View
spec (#11041). We wanted to share our approach early and welcome feedback from the
community and the spec author before going further.
Per the spec, a materialized view is a standard Iceberg view whose ViewVersion carries
an optional
storage-tablefield pointing to a regular Iceberg table holding precomputedquery results. When absent, the entity is a plain logical view — fully backward compatible.
Changes in this PR:
StorageTableIdentifier— new value class (namespace,name, optionalcatalog)ViewVersion— new defaultstorageTable()returning null for plain viewsBaseViewVersion— exposes@Nullable storageTable()for ImmutablesViewVersionParser— serializes/deserializes thestorage-tableJSON fieldSpec Reference
#11041 — Materialized View Spec (thanks @JanKaul and all PMC/community for driving this!)
Proposed Implementation Plan
We are planning to implement the spec incrementally and would love early input on
whether this direction looks right to the community. Here is what we are thinking:
StorageTableIdentifier+storage-tablefield onViewVersionRefreshState+SourceStatedata classes (snapshot summary post-refresh)MaterializedViewinterface extendingViewViewCatalogCREATE MATERIALIZED VIEW,REFRESH,DROP)We are happy to adjust the approach, merge steps, or re-sequence based on feedback.
Test Plan
./gradlew :iceberg-api:test --tests "org.apache.iceberg.view.TestStorageTableIdentifier"./gradlew :iceberg-core:test --tests "org.apache.iceberg.view.TestViewVersionParser"