[SPARK-54119] Support METRIC_VIEW creation on V2 catalogs

[chenwang-databricks]

What changes were proposed in this pull request?

This PR extends metric-view support to DS v2 catalogs by routing CREATE VIEW ... WITH METRICS through the ViewCatalog / RelationCatalog APIs introduced by SPARK-52729. Third-party v2 catalogs that implement ViewCatalog can now host metric views with the same metadata fidelity as session-catalog metric views.

1. V2 metric-view CREATE path -- shared with CreateV2ViewExec. A new CreateV2MetricViewExec and CreateV2ViewExec both extend a new V2CreateViewPreparation trait (which itself extends V2ViewPreparation). The trait owns the shared CREATE-side run(): viewExists short-circuit on IF NOT EXISTS, createOrReplaceView for OR REPLACE, and cross-type collision decoding (ViewAlreadyExistsException -> tableExists -> EXPECT_VIEW_NOT_TABLE.NO_ALTERNATIVE). The metric-view subclass only supplies the metric-view-specific bits (no collation, schema-mode UNSUPPORTED, typed viewDependencies, PROP_TABLE_TYPE = METRIC_VIEW) via two optional hooks on V2ViewPreparation. DataSourceV2Strategy intercepts CreateMetricViewCommand on a non-session catalog and routes to the new exec; the v1 session-catalog path stays in CreateMetricViewCommand.run.

2. First-class METRIC_VIEW table type.

• CatalogTableType.METRIC_VIEW is added alongside EXTERNAL / MANAGED / VIEW.
• TableSummary.METRIC_VIEW_TABLE_TYPE = "METRIC_VIEW" constant for the V2 surface.
• The previous view.viewWithMetrics property hack is removed; CatalogTable.isMetricView checks tableType == METRIC_VIEW directly.
• V1Table.summarizeTableType and V1Table.toCatalogTable(catalog, ident, ViewInfo) translate between the V2 property form and the V1 enum.

3. Repo-wide tableType == VIEW audit. Promoting metric views to a distinct CatalogTableType opens silent regressions wherever existing code branches on VIEW. 14 sites updated to accept METRIC_VIEW alongside VIEW:

• Definite fixes per cloud-fan's review: CatalogTable.toJsonLinkedHashMap (DESCRIBE EXTENDED rows), HiveExternalCatalog.{createTable, alterTable, restoreTableMetadata} (4 sites for Hive-metastore-backed metric views).
• Repo grep: SessionCatalog.isView, InMemoryCatalog.listViews, RelationResolution (streaming-on-view rejection), Analyzer.lookupTableOrView (ResolvedPersistentView wrapping), rules.scala (saveDataIntoView), DataStreamWriter (streaming write), DescribeRelationJsonCommand, AnalyzeColumnCommand, AnalyzePartitionCommand, CommandUtils.analyzeTable, and 3 sites in tables.scala (CREATE TABLE LIKE provider lookup, ALTER TABLE ADD COLUMNS, DESCRIBE PARTITION).
• Explicit rejection: SHOW CREATE TABLE on a metric view (no round-trippable CREATE VIEW ... WITH METRICS form yet) throws the dedicated UNSUPPORTED_SHOW_CREATE_TABLE.ON_METRIC_VIEW error rather than falling through to the VIEW branch and emitting a misleading CREATE VIEW ... that loses the metric-view kind.

4. Drop-command parity.

• DropTableCommand (v1 path) treats both VIEW and METRIC_VIEW as views: DROP TABLE rejects either with wrongCommandForObjectTypeError, and DROP VIEW accepts either.
• V2SessionCatalog.dropTableInternal extends the existing "view rejected from DROP TABLE" guard to cover METRIC_VIEW.
• For non-session v2: DropTableExec's existing viewExists fallback already throws EXPECT_TABLE_NOT_VIEW.NO_ALTERNATIVE when a view sits at a DROP TABLE target -- this works unchanged for metric views since RelationCatalog's default viewExists derives from loadRelation and recognizes MetadataOnlyTable + ViewInfo.
• ResolveSessionCatalog's DropView routing comment is clarified: v2 metric views fall through to DataSourceV2Strategy and ViewCatalog.dropView.

5. Typed view dependencies (ViewInfo.viewDependencies).

• New public DTOs in org.apache.spark.sql.connector.catalog: Dependency (sealed interface with Dependency.table(String...) / Dependency.function(String...) factories), TableDependency, FunctionDependency, DependencyList(Dependency[]).
• TableDependency and FunctionDependency carry the dependency identifier as structural multi-part name parts (record TableDependency(String[] nameParts)), not a single dot-flattened string. Arity is preserved per source so multi-level-namespace V2 catalogs (e.g. Iceberg cat.db1.db2.tbl -> 4 parts) and V1 sources without a catalog component (db.tbl -> 2 parts) round-trip without ambiguity against quoted identifiers containing literal ..
• DependencyList's canonical constructor and accessor defensively clone() the array so consumers can't mutate stored ViewInfo dependency state.
• ViewInfo gains a viewDependencies field and a ViewInfo.Builder.withViewDependencies(...) setter.
• MetricViewHelper.collectTableDependencies walks the analyzed plan and emits structural Seq[Seq[String]] parts (V1 sources via TableIdentifier.nameParts; V2 sources via (catalog.name +: namespace) :+ name).

6. metric_view.* descriptor properties. MetricView.getProperties produces canonical descriptive properties (metric_view.from.type, metric_view.from.name / metric_view.from.sql, metric_view.where) that the create path merges into the view's properties bag, so catalog browsers and tooling can inspect the source/filter without re-parsing the YAML body. Long values are truncated to Constants.MAXIMUM_PROPERTY_SIZE; the Scaladoc on getProperties calls out that metric_view.from.sql is therefore a descriptive value, not a round-trippable representation -- consumers should re-read the YAML body for the full SQL.

7. ViewInfo constructor cleanup. The metric-view-specific PROP_TABLE_TYPE = METRIC_VIEW special case is dropped from the generic ViewInfo constructor in favor of properties().putIfAbsent(...). Callers that want a more specific kind (e.g. METRIC_VIEW) call BaseBuilder.withTableType(...) before build() -- exercised by CreateV2MetricViewExec via the new V2ViewPreparation.tableType hook.

8. ViewHelper.aliasPlan(retainMetadata). The user-specified-column-with-comment branch in aliasPlan previously dropped existing column metadata. A new retainMetadata: Boolean = false parameter merges the analyzed attribute's metadata into the new comment metadata. ViewHelper.prepareTable passes retainMetadata = isMetricView, preserving the per-column metric_view.type / metric_view.expr keys that the analyzer attaches to dimensions and measures even when the user renames columns and adds comments.

9. Error classes.

• New INVALID_METRIC_VIEW_YAML (sqlState 42K0L). MetricViewPlanner.parseYAML's catch blocks now route through QueryCompilationErrors.invalidMetricViewYamlError instead of SparkException.internalError, so a typo in the user's YAML body surfaces as a user-correctable AnalysisException rather than "please contact support".
• New UNSUPPORTED_SHOW_CREATE_TABLE.ON_METRIC_VIEW (sqlState 0A000) for the v1 session-catalog path.

10. Misc.

• MetricViewCanonical.parseSource accepts multipart identifiers (parseMultipartIdentifier) so 3-part catalog.schema.table source references work as AssetSource.

Why are the changes needed?

Before this PR, metric-view DDL only worked against the session catalog: the create path called SessionCatalog.createTable directly, and there was no way for a third-party v2 catalog (Unity Catalog, Hive Metastore catalog, custom REST catalogs, etc.) to own a metric view's lifecycle. SPARK-52729 (#51419) shipped ViewCatalog and RelationCatalog as the public v2 surface for catalog-managed views; metric views are a kind of view and naturally belong on this surface.

Once metric views can live on a v2 catalog, two more constraints surface:

1. Type discriminator. A consumer reading a row through ViewCatalog.loadView needs to know it's a metric view, not a plain SQL view, so it can render the right UI / planner output. Encoding this in PROP_TABLE_TYPE = METRIC_VIEW keeps the distinction wire-compatible and lets V1Table.toCatalogTable reconstruct CatalogTableType.METRIC_VIEW on the read path.
2. Structured dependency lineage. Metric views always reference at least one source table; cataloging that lineage as flat string properties or single dot-joined strings loses arity for multi-level namespaces and is ambiguous against quoted identifiers. A typed DependencyList of TableDependency / FunctionDependency with structural String[] nameParts lets catalogs persist the lineage as a first-class field with full fidelity.

The remaining changes (drop-command parity, aliasPlan metadata retention, metric_view.* properties, parseMultipartIdentifier, tableType == VIEW audit) are mechanical follow-ups that fall out of supporting metric views as a real CatalogTableType and as a v2 catalog citizen -- without them, basic operations like DROP VIEW, DESCRIBE TABLE EXTENDED, or CREATE VIEW (a COMMENT 'c') WITH METRICS ... would silently degrade.

Does this PR introduce any user-facing change?

Yes, both for end users and for catalog plugin developers:

End users:

• CREATE VIEW <ident> WITH METRICS ... now works against any v2 catalog that implements ViewCatalog. Previously it was rejected with MISSING_CATALOG_ABILITY.VIEWS for non-session catalogs. IF NOT EXISTS and OR REPLACE are honored on the v2 path (regression vs. v1 fixed).
• A v2 metric view can be queried with SELECT region, measure(count_sum) FROM <mv> ..., dropped with DROP VIEW, listed via SHOW VIEWS, and described with DESCRIBE TABLE / DESCRIBE TABLE EXTENDED. DROP TABLE on a metric view throws EXPECT_TABLE_NOT_VIEW. SHOW TABLES does not list metric views (per RelationCatalog contract).
• SHOW CREATE TABLE on a metric view throws an explicit unsupported-feature error (no round-trippable form yet).
• Session-catalog metric views are now stored as CatalogTableType.METRIC_VIEW instead of CatalogTableType.VIEW + view.viewWithMetrics=true. Observable in DESCRIBE TABLE EXTENDED's Type row and the tableType column of the tables system table. SQL behavior is unchanged.
• Error messages from DROP TABLE / DROP VIEW mismatch now mention METRIC_VIEW alongside VIEW.
• Malformed metric-view YAML now surfaces as INVALID_METRIC_VIEW_YAML (user-correctable) instead of "Spark internal error, please contact support".

Catalog plugin developers:

• New public API surface in org.apache.spark.sql.connector.catalog: sealed interface Dependency with permits TableDependency, FunctionDependency, both records carrying String[] nameParts. DependencyList(Dependency[]) with defensive copy. All @Evolving, @since 4.2.0.
• ViewInfo gains a typed viewDependencies() accessor and ViewInfo.Builder.withViewDependencies(...) setter.
• TableSummary.METRIC_VIEW_TABLE_TYPE = "METRIC_VIEW" constant.
• CatalogTableType.METRIC_VIEW enum value (v1 surface).
• V2ViewPreparation (private to org.apache.spark.sql.execution.datasources.v2) gains optional viewDependencies / tableType hooks.

How was this patch tested?

MetricViewV2CatalogSuite -- 29 tests across 5 sections, all against an in-memory ViewCatalog test fixture (MetricViewRecordingCatalog extends InMemoryTableCatalog with RelationCatalog):

Section 1 -- CREATE-related (10 tests):

• V2 catalog receives METRIC_VIEW table type and view text via ViewInfo.
• V2 catalog path populates metric_view.* descriptor properties + view context (currentCatalog / currentNamespace) + captured SQL configs.
• V2 catalog path captures SQLSource and comment.
• Metric view columns carry metric_view.type / metric_view.expr in column metadata.
• User-specified column names with comments preserve metric_view.* metadata (pins the aliasPlan(retainMetadata = true) fix).
• CREATE OR REPLACE VIEW ... WITH METRICS replaces an existing v2 metric view (asserts on the replacement's distinguishing fields: queryText, metric_view.where, dependencies).
• CREATE VIEW IF NOT EXISTS ... WITH METRICS is a no-op when the view exists (catalog never sees the second createView call).
• CREATE VIEW ... WITH METRICS over a v2 table at the ident throws EXPECT_VIEW_NOT_TABLE.NO_ALTERNATIVE.
• CREATE VIEW IF NOT EXISTS ... WITH METRICS is a no-op when a v2 table sits at the ident (v1 parity).
• CREATE VIEW ... WITH METRICS on a non-ViewCatalog catalog fails with MISSING_CATALOG_ABILITY.VIEWS.

Section 2 -- Dependency extraction (5 tests):

• SQL source JOIN captures both tables as 3-part nameParts.
• SQL source subquery deduplicates same-table references.
• SQL source self-join deduplicates same-table references.
• V1 session-catalog source emits 2-or-3-part nameParts.
• Multi-level V2 namespace source (testcat.ns_a.ns_b.events_deep) emits 4-part nameParts.

Section 3 -- SELECT cases (5 tests, modeled on MetricViewSuite patterns):

• SELECT measure(count_sum) FROM <mv> GROUP BY region returns aggregated rows (exercises the full loadRelation -> MetadataOnlyTable(ViewInfo) -> V1Table.toCatalogTable(ViewInfo) -> ResolveMetricView round-trip).
• SELECT measure(...) WHERE region = ... -- query-layer filter on top of the view.
• View's pre-defined where clause is applied (where = Some("count > 1") filters at view-resolution time).
• Multiple measures with different aggregations (sum / sum / max).
• ORDER BY measure(...) DESC LIMIT 1 over the metric view.

Section 4 -- DESCRIBE cases (2 tests):

• DESCRIBE TABLE EXTENDED round-trips through loadRelation and emits the View Text / Type rows (gated through the CatalogTable.toJsonLinkedHashMap VIEW gate, which now accepts METRIC_VIEW).
• DESCRIBE TABLE (non-EXTENDED) returns the aliased columns.

Section 5 -- DROP / SHOW cases (7 tests):

• DROP VIEW succeeds on a v2 metric view.
• DROP VIEW IF EXISTS on a non-existent v2 metric view is a no-op.
• DROP TABLE on a v2 metric view throws EXPECT_TABLE_NOT_VIEW.NO_ALTERNATIVE (and asserts the metric view is not deleted).
• DROP TABLE IF EXISTS on a v2 metric view also throws (IF EXISTS doesn't silence the wrong-type error, v1 parity).
• SHOW CREATE TABLE on a v2 metric view throws UNSUPPORTED_FEATURE.TABLE_OPERATION with operation "SHOW CREATE TABLE".
• SHOW TABLES does not list v2 metric views (asserts source events is included, mv is excluded).
• SHOW VIEWS lists v2 metric views.

Existing session-catalog metric-view tests (MetricViewSuite, SimpleMetricViewSuite) and v1 path tests pass unchanged. DDLSuite and HiveDDLSuite had their tableTypes enumerations updated to include METRIC_VIEW in two assertion lists.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Cursor (Claude Sonnet 4.7)

[cloud-fan]

Thanks for extending metric views to v2 catalogs. The shape of the change is right — route through ViewCatalog.createView with PROP_TABLE_TYPE = METRIC_VIEW and a typed viewDependencies, and promote METRIC_VIEW to a first-class CatalogTableType. A few significant concerns:

1. v2 path forks instead of reusing V2ViewPreparation and changes behavior on IF NOT EXISTS / OR REPLACE. CreateMetricViewCommand.createMetricViewInV2Catalog is open-coded inside a v1 RunnableCommand and duplicates most of V2ViewPreparation.buildViewInfo, but with materially different behavior: allowExisting and replace are silently dropped on the v2 path (regression vs. v1, which honored ignoreIfExists), there's no viewExists short-circuit, no createOrReplaceView, no cross-type collision decoding (so a table sitting at the ident leaks ViewAlreadyExistsException instead of unsupportedCreateOrReplaceViewOnTableError), and the CharVarcharUtils / SchemaUtils.checkColumnNameDuplication / checkIndeterminateCollationInSchema / owner-stamping work that V2ViewPreparation does is missing. Refactoring this into a CreateV2MetricViewExec (or extending V2ViewPreparation with a viewDependencies / extraProperties hook) plus a DataSourceV2Strategy rule would close all of these gaps in one move.

2. Audit other tableType == VIEW gates for METRIC_VIEW. Promoting metric views to a distinct CatalogTableType opens up silent regressions wherever existing code branches on VIEW. Two I'm fairly sure about: CatalogTable.toJsonLinkedHashMap (interface.scala:670) gates View Text / View Original Text / View Schema Mode / View Catalog and Namespace / SQL Path on tableType == VIEW, so DESCRIBE TABLE EXTENDED on a metric view will lose all of those rows — visible regression vs. the pre-PR viewWithMetrics storage. And HiveExternalCatalog.{createTable, alterTable, restoreTableMetadata} (lines 277, 297, 598, 838) only special-case VIEW, so a Hive-metastore-backed session-catalog metric view will go through the wrong branches on persist and read-back. Tests don't cover Hive-backed metric views, so this would only surface in production. Worth grepping tableType == CatalogTableType.VIEW and tableType == VIEW repo-wide and deciding which sites also need to accept METRIC_VIEW.

3. TableDependency.tableFullName is a single dot-joined string but the Javadoc claims a fixed three-part shape. This breaks down two ways: (a) MetricViewHelper.collectTableDependencies uses unquotedString for V1 sources, which yields a 2-part db.tbl whenever TableIdentifier.catalog is unset, and mkString(".") for V2 sources, which yields N-part names for catalogs with multi-level namespaces (e.g., Iceberg cat.db1.db2.tbl is 4 parts) — so the producer doesn't honor the contract; (b) any consumer parsing the string back via split/join loses arity and is ambiguous against quoted identifiers containing a literal .. Since this is brand-new public API, it's the right time to make the storage structural — e.g., TableDependency(String[] nameParts) — rather than committing to a string the producer can't always honor.

4. MetricViewPlanner.parseYAML raises SparkException.internalError for malformed user YAML. Replacing invalidLiteralForWindowDurationError is right, but internalError is the wrong category — a typo in the user's YAML body shouldn't surface as "Spark internal error, please contact support." Should be a QueryCompilationErrors / AnalysisException with a real error class.

Minor stuff (typed DTO sealing/mutability, the ViewInfo constructor's metric-view special case, missing tests for OR REPLACE / IF NOT EXISTS / V1-source dependency / multi-level-namespace V2 source / read-back through loadRelation) is in the inline comments below.

[cloud-fan]

This new method open-codes a v2 view-CREATE path inside a v1 RunnableCommand. The plain-view v2 path is already factored: V2ViewPreparation.buildViewInfo() (CreateV2ViewExec.scala:62-110) does almost exactly what this method's lines 109-138 do — alias plan, schema, properties, currentCatalog/Namespace, sqlConfigs, schemaMode, queryColumnNames, comment — and CreateV2ViewExec adds the IF NOT EXISTS / OR REPLACE / cross-type-collision handling.

Would you consider refactoring this into either (a) a CreateV2MetricViewExec extends V2ViewPreparation exec invoked by a DataSourceV2Strategy rule that matches CreateMetricViewCommand on a non-session catalog, or (b) extending V2ViewPreparation with a viewDependencies / extraProperties hook that CreateV2ViewExec ignores and a metric-view subclass populates? Either way the metric-view path picks up the same flag handling and validation as plain v2 views without copy-paste.

[cloud-fan]

allowExisting and replace are silently ignored here:

• CREATE VIEW IF NOT EXISTS … WITH METRICS on a v2 catalog will throw ViewAlreadyExistsException instead of being a no-op — regression vs. v1 (createMetricViewInSessionCatalog passes ignoreIfExists = allowExisting).
• CREATE OR REPLACE VIEW … WITH METRICS will likewise throw instead of replacing.
• If a table sits at the ident (mixed RelationCatalog), the ViewAlreadyExistsException from RelationCatalog.createView will leak as a generic "already exists" instead of the dedicated unsupportedCreateOrReplaceViewOnTableError that CreateV2ViewExec's catch block (CreateV2ViewExec.scala:153-169) decodes via tableExists.

CreateV2ViewExec.run shows the full pattern: viewExists short-circuit on IF NOT EXISTS, then createOrReplaceView vs createView, with the catch block handling the table-collision case. None of the three paths is covered by the new test suite today — see comment there.

[cloud-fan]

Compared with V2ViewPreparation.buildViewInfo for plain v2 views, the v2 metric-view path is missing:

• CharVarcharUtils.getRawSchema(...) on the aliased schema before passing it to the builder.
• SchemaUtils.checkColumnNameDuplication(...).
• SchemaUtils.checkIndeterminateCollationInSchema(query.schema).
• withOwner(CurrentUserContext.getCurrentUser) — v1 (prepareTable → CatalogTable) and plain v2 (CreateV2ViewExec) both stamp the owner, but here the captured ViewInfo has no PROP_OWNER.

All four are exactly what V2ViewPreparation.buildViewInfo already does — the cleanest fix is to share that code rather than re-add these one by one.

[cloud-fan]

Adding METRIC_VIEW as a separate CatalogTableType means every existing tableType == CatalogTableType.VIEW site needs to be audited for whether it should also accept METRIC_VIEW. Two visible regressions I noticed in unchanged code:

(a) CatalogTable.toJsonLinkedHashMap (line 670 of this file). The block emitting "View Text", "View Original Text", "View Schema Mode", "View Catalog and Namespace", and "SQL Path" is gated on tableType == CatalogTableType.VIEW. With this PR's change to tableType = METRIC_VIEW, all of those rows disappear from DESCRIBE TABLE EXTENDED output — a user-visible regression vs. the pre-PR VIEW + viewWithMetrics=true storage. Likely fix: change to if (tableType == VIEW || tableType == METRIC_VIEW).

(b) HiveExternalCatalog. Three sites only handle VIEW:

• createTable (lines 277, 297) — empty-schema fallback for Hive-incompatible types.
• alterTable (line 598) — VIEW-specific alter path.
• restoreTableMetadata (line 838) — read-back falls into restoreHiveSerdeTable instead of restoring schema from view properties.

Tests use the in-memory catalog so the Hive path isn't exercised, but a Hive-metastore-backed session-catalog metric view (the realistic deployment) won't round-trip correctly.

Worth a repo-wide grep for tableType == CatalogTableType.VIEW / tableType == VIEW to enumerate the rest before merging.

[cloud-fan]

The "three-part name in the form catalog_name.schema_name.table_name" claim doesn't hold for catalogs with multi-level namespaces (e.g. Iceberg cat.db1.db2.tbl is 4 parts), and the producer in MetricViewHelper.collectTableDependencies doesn't always emit three parts either: V1 sources use TableIdentifier.unquotedString which yields 2 parts when the catalog component is unset, and V2 sources dot-join the namespace which yields N parts.

Since the dot-flattened string is also ambiguous against quoted identifiers containing a literal ., the cleanest fix is to make this DTO structural — e.g. record TableDependency(String[] nameParts) (and the same for FunctionDependency). Consumers can join the parts themselves if they need a flat string, but they can't recover arity if the join happens at storage time.

[cloud-fan]

Embedding the metric-view-specific value in the generic ViewInfo constructor is awkward — it makes ViewInfo aware of one specific PROP_TABLE_TYPE value that lives in the metric-view code. BaseBuilder.withTableType already exists (TableInfo.java:147); the metric-view path could just call it after withProperties(...) and let this constructor remain the unconditional default:

  private ViewInfo(Builder builder) {
    super(builder);
    ...
    properties().putIfAbsent(TableCatalog.PROP_TABLE_TYPE, TableSummary.VIEW_TABLE_TYPE);
  }

and in metricViewCommands.scala drop the viewProperties.put(...) for PROP_TABLE_TYPE and add .withTableType(TableSummary.METRIC_VIEW_TABLE_TYPE) after .withProperties(viewProperties). That way ViewInfo doesn't reference METRIC_VIEW_TABLE_TYPE at all, and adding future view kinds doesn't require touching this class.

[cloud-fan]

The Javadoc claim "A dependency is one of: TableDependency or FunctionDependency" is only structurally enforced if the interface is sealed. As a plain interface on Java 17, third-party connectors can implement it and the "is one of" claim is no longer true.

Suggested change

	public interface Dependency {
	@Evolving
	public sealed interface Dependency permits TableDependency, FunctionDependency {

(Or alternatively, drop the "is one of" line if you want to leave the interface open for future extension.)

[cloud-fan]

Records auto-generate accessors that return field references, so DependencyList.dependencies() hands callers the live array — a downstream consumer can mutate the lineage list of any ViewInfo via this accessor.

Either defensively copy in the canonical constructor and the accessor, or expose List<Dependency> instead:

public DependencyList {
  Objects.requireNonNull(dependencies, "dependencies must not be null");
  dependencies = dependencies.clone();
}

public Dependency[] dependencies() {
  return dependencies.clone();
}

Same consideration applies to existing ViewInfo.queryColumnNames() / currentNamespace(), but those are pre-existing — at least worth doing it on the new types.

[cloud-fan]

Test gaps worth filling — especially the first three, which would have caught the silent-flag-ignore findings on the create path:

• CREATE OR REPLACE VIEW … WITH METRICS on the v2 catalog (currently throws ViewAlreadyExistsException instead of replacing).
• CREATE VIEW IF NOT EXISTS … WITH METRICS on the v2 catalog (regression vs. v1; currently throws instead of no-op).
• CREATE VIEW … WITH METRICS against a v2 catalog where a table sits at the target ident — should produce the dedicated cross-type-collision error, not a generic ViewAlreadyExistsException.
• A V1 source dependency: a v2 metric view referencing a session-catalog table (HiveTableRelation / LogicalRelation). Surfaces what tableFullName() actually contains for that case.
• A multi-part-namespace V2 source: a metric view whose source identifier is cat.ns1.ns2.tbl. Surfaces what mkString(".") produces for tableFullName().
• Read-back: SELECT * FROM <v2 metric view> (or at least EXPLAIN / DESCRIBE EXTENDED) so the loadRelation → MetadataOnlyTable → V1Table.toCatalogTable(ViewInfo) round-trip is exercised end-to-end. The capture-only fixture pins what flows INTO createView but not what comes back OUT.

[cloud-fan]

Worth noting in the Scaladoc that metric_view.from.sql is truncated to Constants.MAXIMUM_PROPERTY_SIZE and is therefore a descriptive value (for catalog browsers / lineage tooling), not a round-trippable representation of the source — a consumer relying on it to reconstruct the view body would silently get a truncated query for any view whose source SQL exceeds the limit.

[cloud-fan]

Round 2 review.

Status: 13 prior findings addressed (4 main concerns + 9 inline), 0 remaining, 4 new (3 newly introduced in commits since the prior review at 8f62efc6, 1 late catch). The shared V2ViewPreparation / V2CreateViewPreparation refactor reads cleanly, the typed Dependency / DependencyList API replaces the dot-joined-string form, the INVALID_METRIC_VIEW_YAML error is in place, and the tableType == VIEW audit covers most sites. Four follow-ups inline.

[cloud-fan]

verifyAlterTableType at line 1086 of this file was missed in the audit. With tableType == METRIC_VIEW:

• ALTER TABLE <metric_view> RENAME TO ... falls through both arms (line 1092 case CatalogTableType.VIEW if !isView and line 1096 case o if o != CatalogTableType.VIEW && isView) and silently renames the metric view as if it were a regular table — regression vs. the pre-PR viewWithMetrics storage where the same call threw cannotAlterViewWithAlterTableError.
• ALTER VIEW <metric_view> RENAME TO ... matches case o if o != CatalogTableType.VIEW && isView and throws cannotAlterTableWithAlterViewError ("cannot alter table with alter view") — wrong category for a view-kind.

Same fix pattern as the DROP audit on this hunk: widen both cases at line 1092 / 1096 to also accept METRIC_VIEW.

[cloud-fan]

The most recent commit (d55143f5b6184d2638e7fde4dae1fc29040b9509) adds three new public APIs (StructField.json, StructField.fromJson, Column.fromStructField) and widens DataType.parseStructField from private to private[sql]. None of them are called by code in this PR — verified by grep, the only call sites are the tests added in the same commit. Per the commit message they exist to support a downstream Unity Catalog connector.

Public API additions to org.apache.spark.sql.types and org.apache.spark.sql.connector.catalog warrant a dedicated PR with its own JIRA ticket so they can be reviewed independently of metric-view support. Could you split this commit out?

[cloud-fan]

This diverges from the canonical inverse CatalogV2Util.structFieldToV2Column (CatalogV2Util.scala:762), which strips the "comment" key from f.metadata().json() before stamping it as metadataInJSON — because the comment is exposed separately via Column.comment().

Here, when field was built with withComment(...), the "comment" key sits inside field.metadata() and gets written into metadataInJSON as well, while also being lifted into comment. Net effect: the comment is duplicated. Round-trip via v2ColumnToStructField happens to recover, but any downstream consumer reading metadataInJSON will see an unexpected "comment" entry.

Mirror structFieldToV2Column and strip "comment" before serializing — using a MetadataBuilder to drop the key, then null out the result if it becomes empty. That also matches the existing function's behavior of returning null metadataInJSON for fields whose only metadata was the comment.

[cloud-fan]

ident.asTableIdentifier throws requiresSinglePartNamespaceError whenever ident has a multi-level namespace. Plain v2 views (CreateV2ViewExec, this file:324) carry Identifier directly and support multi-level namespace targets, but v2 metric views fail here because MetricViewHelper.analyzeMetricViewText requires a TableIdentifier.

The dependency-extraction multi-level test only exercises the source (testcat.ns_a.ns_b.events_deep); the metric view's own ident is always single-level (testcat.ns.mv) — a CREATE at cat.db1.db2.mv would fail here. analyzeMetricViewText only uses the name to (a) build a synthetic CatalogTable for analysis context and (b) feed name.nameParts into verifyTemporaryObjectsNotExists, both of which could take Seq[String] directly.

Either lift the helper to multi-part nameParts or call out the single-level-namespace constraint as a known limitation of the v2 metric-view path.