Skip to content

[SPARK-58895][SQL] Route custom session view catalogs through V2 commands - #58161

Open
manuzhang wants to merge 2 commits into
apache:masterfrom
manuzhang:codex/route-session-view-catalog
Open

[SPARK-58895][SQL] Route custom session view catalogs through V2 commands#58161
manuzhang wants to merge 2 commits into
apache:masterfrom
manuzhang:codex/route-session-view-catalog

Conversation

@manuzhang

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR routes view commands through V2 execution when a custom ViewCatalog is installed as
spark_catalog.

The change:

  • makes ResolveSessionCatalog use V1 view commands only for session catalogs that do not
    implement ViewCatalog, while preserving V1 handling for delegated V1View instances;
  • routes native custom-catalog view creation, alteration, inspection, metric-view creation, and
    removal through the existing V2 commands;
  • extends SHOW VIEWS to combine native V2 views with delegated V1 views and local/global
    temporary views, including filtering, de-duplication, multi-part namespaces, and namespace
    error handling; and
  • adds coverage for native, delegated, and temporary views in a custom session ViewCatalog.

Why are the changes needed?

Spark currently identifies spark_catalog as the session catalog and rewrites its view commands
to V1 commands even when the configured implementation supports ViewCatalog. This bypasses the
custom catalog's view implementation and makes commands such as CREATE VIEW, ALTER VIEW,
DROP VIEW, and SHOW VIEWS behave inconsistently.

The dispatch needs to consider the catalog's view capability and the resolved view payload so
native V2 views use the custom catalog while delegated V1 and temporary views retain their
existing behavior.

Does this PR introduce any user-facing change?

Yes. When spark_catalog is configured with a custom ViewCatalog, persistent view DDL and
inspection commands now operate through that catalog. Delegated V1 views and temporary views
remain accessible, including through SHOW VIEWS. Session catalogs without ViewCatalog keep
their existing V1 behavior.

How was this patch tested?

Added tests to DataSourceV2MetadataViewSuite covering:

  • V2 view and metric-view commands when a ViewCatalog is installed as spark_catalog;
  • delegated V1 views alongside native V2 and temporary views;
  • SHOW VIEWS filtering, custom-only and multi-part namespaces, and persistent/global-temp name
    collisions; and
  • rejection of V1-only namespaces hidden by the custom catalog.

No tests were run locally, per the repository instructions for this session. git diff --check
passes.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Codex (GPT-5)

manuzhang and others added 2 commits August 20, 2026 15:31
@manuzhang
manuzhang marked this pull request as ready for review August 20, 2026 08:38
@manuzhang

Copy link
Copy Markdown
Member Author

@szehon-ho @aokolnychyi This is another gap discovered while working on apache/iceberg#14984. Please help review, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant