Skip to content

Spark: Extend V2 strategy support for view-related commands #17577

Description

@GGraziadei

Follow-up from #14984 (comment).

What happens

ResolveViews resolves any UnresolvedTableOrView pointing at an Iceberg V2 view into a ResolvedV2View:

case u @ UnresolvedTableOrView(CatalogAndIdentifier(catalog, ident), _, _, _) =>
  ViewUtil.loadView(catalog, ident).map(view => ResolvedV2View(...)).getOrElse(u)

RewriteViewCommands then rewrites the view commands Iceberg owns, at parse time, into Iceberg-aware nodes: CreateView, DropView, RenameTable, DescribeRelation, ShowCreateTable, ShowTableProperties, ShowViews, and UnresolvedView. For those, ExtendedDataSourceV2Strategy has a matching case and planning succeeds.

Commands outside that set — SHOW COLUMNS, ANALYZE TABLE, and similar — still resolve to a ResolvedV2View, but the strategy has no case for a bare ResolvedV2View and falls through to case _ => Nil (ExtendedDataSourceV2Strategy.scala:193). The result is a planner failure rather than a clear unsupported-operation error.

Scope

This is not specific to the Spark 4.2 work. The same UnresolvedTableOrView case exists in spark/v4.0 and spark/v4.1 (ResolveViews.scala:59 in both), so any Spark module supporting Iceberg V2 views is affected. It reflects existing Spark-side V2 view limitations, and was deliberately left out of scope in #14984.

Temporary views are not affected: they are resolved before the extension rule runs, as established in the PR discussion.

Proposed work

  • Decide, per command, whether Iceberg should add V2 strategy support or surface a clear unsupported-operation error naming the command and the view.
  • Apply the outcome consistently across the spark/v4.x modules rather than only the newest one.
  • Add coverage in TestViews, which currently exercises neither SHOW COLUMNS nor ANALYZE TABLE against a V2 view.

cc @manuzhang

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions