Skip to content

[SPARK-56501] SET PATH syntax#55364

Closed
srielau wants to merge 13 commits intoapache:masterfrom
srielau:SPARK-56501-SET-PATH-syntax
Closed

[SPARK-56501] SET PATH syntax#55364
srielau wants to merge 13 commits intoapache:masterfrom
srielau:SPARK-56501-SET-PATH-syntax

Conversation

@srielau
Copy link
Copy Markdown
Contributor

@srielau srielau commented Apr 16, 2026

What changes were proposed in this pull request?

(SPARK-56501): SET PATH command with structured session state on CatalogManager

  • spark.sql.path.enabled (default false): master switch for the PATH feature.
  • Session path stored as structured Option[Seq[Seq[String]]] on CatalogManager (not serialized in SQLConf). This makes the "only settable via SET PATH" invariant structural — spark.conf.set/RESET/RuntimeConfig cannot bypass it.
  • SET PATH = schema1, schema2, ... with shortcuts:
    • DEFAULT_PATH: expands to the legacy resolutionSearchPath entries (builtin, session, current_schema) ordered by spark.sql.function.resolutionOrder. Composable with other elements, e.g. SET PATH = DEFAULT_PATH, extra.schema.
    • SYSTEM_PATH: expands to system.builtin, system.session (respects spark.sql.function.resolutionOrder).
    • CURRENT_SCHEMA/CURRENT_DATABASE: stored as virtual system.current_schema marker; expands dynamically so later USE SCHEMA is reflected.
    • PATH: appends to the current stored path.
  • Schema references require at least 2 parts (catalog.namespace); multi-level namespaces (e.g. iceberg_cat.db1.db2) are accepted.
  • Duplicate detection respects spark.sql.caseSensitive and handles multi-level namespaces.
  • CURRENT_PATH() reflects the stored session path when PATH is enabled; falls back to resolutionSearchPath otherwise.
  • Cloned sessions inherit the parent session's path via CatalogManager.copySessionPathFrom.

Resolution is not yet wired to use the stored path. Tables and functions still resolve via the legacy single-schema path. The resolution engine rewiring comes in a follow-up PR.

Why are the changes needed?

SET PATH is a SQL-standard feature (SQL:2023) that gives users control over the order in which unqualified object names are resolved across schemas. This PR adds the command syntax and session state management; resolution will be wired separately.

Part of SPARK-54810.

Does this PR introduce any user-facing change?

Yes.

  • New SET PATH command (requires spark.sql.path.enabled = true).
  • SET PATH = DEFAULT_PATH resets the path to the session default.

How was this patch tested?

  • SetPathSuite (new, 22 tests): SET PATH disabled/enabled, DEFAULT_PATH (standalone + combined + duplicate), SYSTEM_PATH, CURRENT_SCHEMA/CURRENT_DATABASE expansion, PATH append, duplicate detection (plain, after CURRENT_SCHEMA expansion, cross-alias current_database/current_schema, SYSTEM_PATH twice), unqualified (1-part) rejection, multi-level namespace acceptance, backtick-quoted identifier round-trip, case preservation, programmatic SET has no effect, cloned session inherits path.
  • ParametersSuite: 2 tests for SET PATH with IDENTIFIER() parameter markers.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.6

@srielau srielau force-pushed the SPARK-56501-SET-PATH-syntax branch 5 times, most recently from 87b1066 to 8378057 Compare April 16, 2026 17:51
srielau added 2 commits April 16, 2026 10:52
…ated grammar keywords

Add the CURRENT_PATH() builtin function that returns the current SQL
resolution search path as a comma-separated string of qualified schema
names (e.g. "system.builtin,system.session,spark_catalog.default").

Also register the grammar keywords needed by the upcoming SQL PATH
feature (CURRENT_PATH, CURRENT_SCHEMA, CURRENT_DATABASE, DEFAULT_PATH,
SYSTEM_PATH, PATH). CURRENT_PATH and CURRENT_SCHEMA are reserved in
ANSI mode per SQL:2023; the others are non-reserved.

In non-ANSI mode, CURRENT_PATH, CURRENT_DATABASE, and CURRENT_SCHEMA
always resolve to their respective expressions (not UnresolvedAttribute),
matching the behavior of CURRENT_CATALOG.

This is part 1 of the SQL PATH feature (SPARK-54810), split out to keep
the review scope manageable.
…ation

Add the SET PATH command and the SQLConf infrastructure for the SQL
standard PATH feature:

- spark.sql.path.enabled: master switch for the PATH feature
- spark.sql.session.path: internal session path (set only via SET PATH)
- SET PATH = schema1, schema2, ...: set the session path with support
  for shortcuts DEFAULT_PATH, SYSTEM_PATH, CURRENT_SCHEMA, PATH (append)
- CURRENT_PATH() now reflects the stored session path when PATH is enabled
- Reject direct SET of spark.sql.session.path (must use SET PATH)
- Duplicate path entry detection with DUPLICATE_SQL_PATH_ENTRY error
- Three-part schema references rejected with INVALID_SQL_PATH_SCHEMA_REFERENCE
- SET PATH when disabled raises UNSUPPORTED_FEATURE.SET_PATH_WHEN_DISABLED

Resolution is not yet wired to use the stored path -- tables and
functions still resolve via the legacy single-schema path. The
resolution engine rewiring comes in a follow-up PR.

This is part 2a of the SQL PATH feature (SPARK-54810).
@srielau srielau force-pushed the SPARK-56501-SET-PATH-syntax branch from 8378057 to 8a5eca8 Compare April 16, 2026 17:52
# Conflicts:
#	sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
#	sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
#	sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
#	sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
#	sql/core/src/test/scala/org/apache/spark/sql/FunctionQualificationSuite.scala
Copy link
Copy Markdown
Contributor

@cloud-fan cloud-fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Prior state and problem. Before this PR, Spark computed the search path implicitly via SQLConf.resolutionSearchPath(currentCatalog +: currentNamespace), ordering builtin/session/current-catalog by spark.sql.function.resolutionOrder ("first" / "second" / "last"). Users had no way to customize the order or extend the path, which is what SQL:2023's SET PATH is for.

Design approach. The PR adds (1) a feature flag spark.sql.path.enabled, (2) an internal string-valued session config spark.sql.session.path storing the user-set path, (3) a SET PATH = … command whose run() expands shortcuts (DEFAULT_PATH, SYSTEM_PATH, PATH, CURRENT_SCHEMA/CURRENT_DATABASE) and validates duplicates before writing the config, and (4) a currentPathString helper on SQLConf that ReplaceCurrentLike calls for CURRENT_PATH(). When PATH is enabled and a path is stored, CURRENT_PATH() returns the stored list with system.current_schema expanded live against currentCatalog + currentNamespace; otherwise it falls back to legacy resolutionSearchPath. Resolution itself is not yet wired — that comes in a follow-up.

Key design decisions made by this PR. (a) Storing the path as a serialized string in SQLConf instead of structured state on CatalogManager. (b) Guarding "only settable via SET PATH" with an error check inside SetCommand. (c) Using a system.current_schema sentinel so the "live current schema" entry reflects later USE SCHEMA. (d) Accepting 1-part SchemaInPath entries and qualifying them at SET time against the current catalog, remapping unqualified builtin/session to SYSTEM, and rejecting ≥3-part entries.

Main concerns.

  • (a) is not justified in the PR description. CatalogManager already holds analogous per-session state (currentCatalog, currentNamespace) with full propagation and reset support, so the reuse argument for SQLConf is thinner than it appears. Structured storage would eliminate the serialization format (below), remove the need for the guard, and make the invariant structural instead of policed.
  • The string serialization uses . and , as unescaped separators, so backtick-quoted identifiers containing either character round-trip incorrectly — and, more importantly, multi-level namespaces cannot round-trip at all: "cat.db1.db2" parses back as Seq("cat", "db1.db2"). Given Spark supports multi-level namespaces, this is not an edge case.
  • The guard in SetCommand covers only SQL SET; RESET and spark.conf.set/unset write/clear the config directly, so the invariant doesn't actually hold.
  • (d) has two problems. The ≥3-part rejection is wrong — Spark supports multi-level namespaces like iceberg_cat.db1.db2.schema, which this feature should accept. And the 1-part form is a SET-time capture trap: it's pinned to whatever catalog was active at SET PATH time and silently frozen against later USE CATALOG. CURRENT_SCHEMA/CURRENT_DATABASE already cover the dynamic case and SYSTEM_PATH covers the SYSTEM shortcut, so 1-part adds no functional value. If 1-part is disallowed, parts.head is the catalog and parts.tail is the namespace by convention — qualifySchemaParts collapses to an arity check, and the inline qualification and builtin/session → SYSTEM remap both go away.

Also flagging a couple of smaller items that didn't fit inline: SET PATH = …, DEFAULT_PATH silently drops the DEFAULT_PATH (it expands to Seq.empty), and SET PATH = DEFAULT_PATH, DEFAULT_PATH silently succeeds while SYSTEM_PATH, SYSTEM_PATH correctly errors — duplicate handling should be consistent across shortcuts. And SYSTEM_PATH always produces (builtin, session) regardless of spark.sql.function.resolutionOrder, which diverges from legacy resolutionSearchPath — worth resolving before resolution wiring lands.

Comment thread sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala Outdated
Comment thread sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala Outdated
Comment thread sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala Outdated
Comment thread sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala Outdated
Comment thread sql/core/src/test/scala/org/apache/spark/sql/SetPathSuite.scala
srielau added 2 commits April 17, 2026 08:19
…ion, DEFAULT_PATH

Address cloud-fan's review on PR apache#55364:

- Disallow 1-part schema refs (SET-time capture trap); accept multi-level
  namespaces (≥2 parts) as-is — qualifySchemaParts collapses to arity check
- Fix lossy serialization: use \u001E/\u001F control separators so multi-level
  namespaces and backtick-quoted identifiers with ./,  round-trip correctly
- Restrict DEFAULT_PATH to sole-element form (reject when combined)
- SYSTEM_PATH now respects spark.sql.function.resolutionOrder
- Rename deduped → foreach validation; duplicate key handles Seq[String]
- Use idiomatic ctx.pathElement().asScala.map; revert unrelated formatting
- Fix QueryCompilationErrors doc (sqlResolutionPathEntries → resolutionSearchPath)
- Fix SQLConf doc ("session catalog" → "current catalog", "to" → "into")
- Strengthen tests: exact path assertions, cross-alias dup, multi-level ns,
  DEFAULT_PATH combined rejection, backtick round-trip, 1-part rejection
Move the SQL PATH session state from a serialized string config
(spark.sql.session.path) to a structured field on CatalogManager,
alongside currentCatalog/currentNamespace.

- Add _sessionPath: Option[Seq[Seq[String]]] to CatalogManager with
  synchronized getter/setter/clear/reset/copyFrom/currentPathString
- Wire session path cloning in BaseSessionStateBuilder
- SetPathCommand now writes CatalogManager directly
- ReplaceCurrentLike reads catalogManager.currentPathString
- Remove SESSION_PATH config, sessionPath, effectivePathEntries,
  parseSessionPath, formatSessionPath, separators from SQLConf
- Remove SetCommand guard and SET_PATH_VIA_SET error condition
- Add tests: programmatic SET has no effect, cloned session inherits path
@srielau
Copy link
Copy Markdown
Contributor Author

srielau commented Apr 17, 2026

@cloud-fan Thank you for the thorough review. All comments addressed in two commits:

Commit 1 — direct fixes:

  • qualifySchemaParts simplified: 1-part rejected, ≥2-part (multi-level namespaces) accepted as-is
  • dedupedforeach validation; duplicate key now handles Seq[String]
  • DEFAULT_PATH restricted to sole-element form
  • SYSTEM_PATH respects sessionFunctionResolutionOrder
  • Idiomatic asScala.map; reverted unrelated formatting
  • Fixed QueryCompilationErrors doc, SQLConf doc ("session catalog" → "current catalog")
  • Strengthened all test assertions (exact ===), added cross-alias dup, multi-level ns, DEFAULT_PATH combined, backtick round-trip, 1-part rejection tests

Commit 2 — structural redesign per your preferred approach:

  • Moved session path from SQLConf string to structured Option[Seq[Seq[String]]] on CatalogManager
  • Removed SESSION_PATH config, parseSessionPath/formatSessionPath, serialization separators, SetCommand guard, SET_PATH_VIA_SET error
  • SetPathCommand writes CatalogManager directly; ReplaceCurrentLike reads catalogManager.currentPathString
  • Session cloning wired via BaseSessionStateBuilder (copySessionPathFrom)
  • Added test: programmatic spark.conf.set has no effect; cloned session inherits path

Regarding your summary concerns:

  • (a) Storing path as structured CatalogManager state — done.
  • Lossy serialization — eliminated entirely (no serialization).
  • Guard bypass — structurally impossible (config key removed).
  • (d) 1-part rejected, ≥3-part accepted, qualifySchemaParts collapsed to arity check.
  • DEFAULT_PATH inconsistency — fixed (sole-element restriction).
  • SYSTEM_PATH ordering — now respects spark.sql.function.resolutionOrder.

Make DEFAULT_PATH expand to the same entries as the legacy
resolutionSearchPath (builtin, session, current_schema) ordered by
spark.sql.function.resolutionOrder, using the virtual
system.current_schema marker so USE SCHEMA is still reflected.

This allows DEFAULT_PATH to compose with other path elements:
  SET PATH = DEFAULT_PATH, spark_catalog.extra
Previously DEFAULT_PATH was restricted to sole-element use.
Copy link
Copy Markdown
Contributor

@cloud-fan cloud-fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major architecture change since the prior round — structured state on CatalogManager replaces the serialized SQLConf string + SetCommand guard. That resolves the prior serialization round-trip, RESET/programmatic bypass, and multi-level namespace issues cleanly. A handful of new items from this round inline.

Comment thread common/utils/src/main/resources/error/error-conditions.json Outdated
Comment thread sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala Outdated
Comment thread sql/core/src/test/scala/org/apache/spark/sql/SetPathSuite.scala
srielau added 2 commits April 17, 2026 15:13
… path helpers to CatalogManager

- Remove unused sqlConf parameter from ReplaceCurrentLike (and conf
  arg in Optimizer.scala); currentPathString is on CatalogManager now.
- Remove unused currentNamespaceSeq in ReplaceCurrentLike.
- Synchronize _sessionPath read in CatalogManager.currentPathString.
- Move SESSION_PATH_VIRTUAL_CURRENT_SCHEMA, isVirtualCurrentSchemaPathEntry,
  concreteSessionPathEntry, expandSessionPathMarkers from SQLConf to
  CatalogManager companion (they are session-path helpers, not config).
- Fix DUPLICATE_SQL_PATH_ENTRY message for multi-level namespaces.
Copy link
Copy Markdown
Contributor

@cloud-fan cloud-fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-2 items all addressed cleanly — data race, ReplaceCurrentLike unused param, error message wording, helper relocation, and the two new test cases. clearSessionPath() dead branch was declined and I'll leave that to author judgment. The round-2 architecture — structured Option[Seq[Seq[String]]] on CatalogManager with a virtual system.current_schema marker — is unchanged.

A handful of new items this round:

  • Ordering logic duplicated between SetPathCommand.expandPathElements and SQLConf.resolutionSearchPath. Both encode the sessionFunctionResolutionOrder branches for DEFAULT_PATH/SYSTEM_PATH. A shared helper that returns the default ordering as Seq[Seq[String]] (with the current-schema slot supplied by the caller) would keep them from drifting. Tests also don't vary sessionFunctionResolutionOrder to "first" or "last", so any divergence in those branches would currently slip through. Given resolution wiring is a follow-up, locking these branches down with coverage now seems worth it.

Other items inline.

Comment thread sql/core/src/test/scala/org/apache/spark/sql/SetPathSuite.scala
Comment thread sql/core/src/test/scala/org/apache/spark/sql/SetPathSuite.scala
…c fixes, new tests

- Fix visitSetPath Scaladoc placement (was stealing visitSetTimeZone doc)
- Remove isSystemSessionPathEntry from this PR (no call sites; will be
  added in the persist-path PR that needs it)
- Quote parts containing dots in DUPLICATE_SQL_PATH_ENTRY error message
- Extract defaultPathOrder/systemPathOrder on SQLConf to eliminate
  ordering logic duplication between SetPathCommand and resolutionSearchPath
- Add tests for sessionFunctionResolutionOrder = "first" and "last"
- Note that cloneSession shares CatalogManager (same as USE CATALOG/SCHEMA)
Copy link
Copy Markdown
Contributor

@cloud-fan cloud-fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-3 items all addressed cleanly — shared ordering helper, resolution-order tests, Scaladoc relocation, and the backtick quoting in the duplicate error message.

Three substantive items this round:

  • PathElement.PathRef expands to Seq.empty when no path is stored, so SET PATH = PATH, x.y on an enabled-but-unset session silently clobbers the defaults. Once resolution wires up in the follow-up, builtin functions would stop resolving. Inline on SetPathCommand.
  • resolutionSearchPath(Seq.empty) now returns a trailing empty-Seq entry instead of the 2-entry system-only path its Scaladoc still promises. Latent (no current caller passes empty) but worth fixing while the refactor is fresh. Inline on SQLConf.
  • One design question about _sessionPath storage (raw Seq[Seq[String]] + system.current_schema sentinel) — keeping the PathElement-style ADT for storage would delete the four helper functions that exist only to encode/decode the sentinel. Not blocking; raising before more consumers accrete the pattern.

Plus two minor items: PATH_ENABLED doc says "no effect" when disabled but the command throws, and a cosmetic double-space in the grammar alignment.

Comment thread sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala Outdated
Comment thread sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4 Outdated
srielau added 2 commits April 21, 2026 08:36
…onSearchPath guard, doc/grammar fixes

- PathRef on unset session now expands to DEFAULT_PATH entries (prevents
  clobbering defaults when composing PATH with other elements)
- resolutionSearchPath(Seq.empty) now correctly returns system-only path
  instead of trailing empty-Seq entry
- Fix doc: "has no effect" → "is rejected" for PATH_ENABLED
- Fix grammar: align #setPath label column
- Add test: SET PATH = PATH on unset session includes builtin defaults
Copy link
Copy Markdown
Contributor

@cloud-fan cloud-fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-4 items all addressed cleanly — PathRef fallback, resolutionSearchPath(Seq.empty) guard, PATH_ENABLED doc wording, and grammar alignment.

Three late catches from earlier rounds that I missed (none are regressions introduced by round 4 — non-blocking, feel free to fold into a follow-up):

  • SetPathSuite.scala:370-372 — trailing Note incorrectly claims cloneSession() shares the parent's CatalogManager. The claim originated in my round-3 exchange on clone independence; I took it at face value instead of cross-checking BaseSessionStateBuilder.catalogManager. Suggest replacing with a TODO — the real question (what CatalogManager state should propagate on clone) is out of scope here.
  • CatalogManager.scala:254concreteSessionPathEntry Scaladoc understates the method's role; it's also the engine for expandSessionPathMarkers used from currentPathString at CURRENT_PATH() display time.
  • SetPathSuite.scala:339 — test name claims "resolution is case-insensitive" but the body only checks case preservation, and the normalize/toLowerCase dedup path in SetPathCommand.run is uncovered.

Approving.

Comment thread sql/core/src/test/scala/org/apache/spark/sql/SetPathSuite.scala Outdated
Comment thread sql/core/src/test/scala/org/apache/spark/sql/SetPathSuite.scala Outdated
srielau added 2 commits April 21, 2026 13:47
Replace the string-based `system.current_schema` sentinel with a typed
sealed trait `SessionPathEntry` on CatalogManager:

- CurrentSchemaEntry: marker that expands dynamically with USE SCHEMA
- LiteralPathEntry(parts): a fully qualified schema reference

This eliminates SESSION_PATH_VIRTUAL_CURRENT_SCHEMA,
isVirtualCurrentSchemaPathEntry, concreteSessionPathEntry, and
expandSessionPathMarkers. Consumers now pattern-match on the ADT.

The session path type changes from Option[Seq[Seq[String]]] to
Option[Seq[SessionPathEntry]]. resolvePathEntries replaces
expandSessionPathMarkers for converting to concrete parts.
Copy link
Copy Markdown
Contributor

@dtenedor dtenedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this feature, LGTM

@dtenedor dtenedor closed this in 345b856 Apr 21, 2026
@dtenedor
Copy link
Copy Markdown
Contributor

LGTM, merging to master

cloud-fan pushed a commit that referenced this pull request Apr 23, 2026
…e in DESCRIBE

### What changes were proposed in this pull request?

When `spark.sql.path.enabled` is true, persist the effective resolution path at creation time for views and SQL functions, and expose it in DESCRIBE output.

**View persistence (`views.scala`):**
- Store the frozen path as JSON in `view.resolutionPath` property at CREATE VIEW time.
- Persisted views strip `system.session` from the stored path since persistent views cannot reference temporary objects (session scope). Temporary views keep it because temp objects can reference other temp objects.

**SQL function persistence (`CreateSQLFunctionCommand.scala`, `SQLFunction.scala`):**
- Store the frozen path as JSON in `function.resolutionPath` property at CREATE FUNCTION time.
- Same stripping rules as views.

**Storage format:**
- Path entries are serialized as JSON arrays: `[["spark_catalog","default"],["system","builtin"]]`
- This naturally supports multi-level namespaces (nested schemas), e.g. `[["catalog","ns1","ns2"]]`.

**DESCRIBE output for views (`interface.scala`):**
- `DESCRIBE EXTENDED` / `DESCRIBE FORMATTED` shows `SQL Path` with backtick-quoted entries (e.g. `` `spark_catalog`.`default`, `system`.`builtin` ``).
- `DESCRIBE ... AS JSON` exposes `sql_path` as an array of objects, consistent with `describeIdentifier`:
  ```json
  "sql_path": [
    {"catalog_name": "spark_catalog", "namespace": ["default"]},
    {"catalog_name": "system", "namespace": ["builtin"]}
  ]
  ```

**DESCRIBE FUNCTION EXTENDED for SQL UDFs (`functions.scala`):**
- Shows `SQL Path` line for SQL UDFs when PATH is enabled and a stored path exists.
- For persistent functions, reads from catalog metadata; for temp SQL UDFs, extracts from the usage JSON blob.

**Backward compatibility:**
- Old views/functions created without PATH enabled have no `resolutionPath` property. They fall back to the default resolution path (`resolutionSearchPath` with session function resolution order).

The stored path is **not yet used during analysis** -- frozen path resolution comes in a follow-up PR. This PR only persists and displays the metadata.

### Why are the changes needed?

Views and SQL functions need to capture the resolution path at creation time so that their body can later resolve with the same path, independent of the caller's session path. This is the SQL-standard behavior for view and routine body resolution.

Part of [SPARK-54810](https://issues.apache.org/jira/browse/SPARK-54810). Depends on [SPARK-56501](#55364) (SET PATH syntax).

### Does this PR introduce _any_ user-facing change?

Yes.
- `DESCRIBE EXTENDED` on views shows `SQL Path` when PATH is enabled.
- `DESCRIBE FUNCTION EXTENDED` on SQL UDFs shows `SQL Path` when PATH is enabled.
- `DESCRIBE ... AS JSON` includes `sql_path` field for views when PATH is enabled.
- View and SQL function metadata now includes the frozen resolution path.

### How was this patch tested?

- `DescribeTableSuite`: new test verifying `DESCRIBE EXTENDED view AS JSON` includes `sql_path` with correct structure, and regular `DESCRIBE EXTENDED` shows `SQL Path` (both V1 catalog V1 and V2 command variants).
- `DescribeTableSuiteBase`: added `SqlPathEntry` case class and `sql_path` field to `DescribeTableJson` for JSON deserialization.
- `SetPathSuite`: all 22 existing tests pass (no regressions).
- Compiled and tested locally with SBT.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.6

Closes #55383 from srielau/SPARK-56520-persist-path.

Authored-by: Serge Rielau <serge@rielau.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants