[SPARK-53840][SQL][FollowUp] Make SHOW TABLE EXTENDED AS JSON match V2 metadata#57362
Closed
linhongliu-db wants to merge 2 commits into
Closed
Conversation
linhongliu-db
marked this pull request as ready for review
July 20, 2026 18:10
linhongliu-db
force-pushed
the
task/spark-53840-show-table-extended-json-v2-metadata/implementation
branch
from
July 20, 2026 20:56
1e94281 to
57f4a03
Compare
uros-b
approved these changes
Jul 21, 2026
uros-b
left a comment
Member
There was a problem hiding this comment.
LGTM, thank you @linhongliu-db! cc @cloud-fan @urosstan-db
cloud-fan
approved these changes
Jul 21, 2026
cloud-fan
left a comment
Contributor
There was a problem hiding this comment.
0 blocking, 0 non-blocking, 0 nits.
The implementation and regression coverage are consistent with the V2 catalog contracts; no actionable findings.
Verification
Traced both V2 catalog branches through listRelationSummaries / listTableSummaries, verified pattern filtering remains on relation names, compared the relation path with ShowTablesExec, and checked the TableCatalog and RelationCatalog default summary contracts. No tests were run as part of this review.
cloud-fan
pushed a commit
that referenced
this pull request
Jul 21, 2026
…2 metadata ### What changes were proposed in this pull request? This follow-up updates `SHOW TABLE EXTENDED ... AS JSON` for V2 catalogs to use catalog-provided summary metadata when producing extended JSON rows. - For V2 relation catalogs, extended JSON now uses `RelationCatalog#listRelationSummaries`, so relations such as views are preserved. - For V2 table catalogs, extended JSON now uses `TableCatalog#listTableSummaries`. - The extended JSON `type` field now comes from `TableSummary.tableType()` instead of being hard-coded to `TABLE`. - V2 command-suite coverage now checks the V2 summary table type and covers a relation catalog returning both a table and a view. ### Why are the changes needed? `SHOW TABLE EXTENDED` text output uses V2 summary metadata for table type information, but the JSON path was falling back to identifier-only table listing and hard-coding the type as `TABLE`. That loses V2 table type metadata, and for relation catalogs it can also miss V2 views that are available through relation summaries. ### Does this PR introduce _any_ user-facing change? Yes. On unreleased Spark master, `SHOW TABLE EXTENDED ... AS JSON` for V2 catalogs now reports the catalog-provided table or view type metadata and preserves V2 relation-catalog views in the JSON output. ### How was this patch tested? Added regression coverage in `ShowTablesSuite`. Also checked: ``` git diff --check apache/master..HEAD ``` GitHub PR checks passed on head `39f507910201d5dd08bbdb58125e1f7c0ac3809a`: - `Build` - `Notify test workflow` Attempted local Spark SQL suite validation, but the environment could not download required build dependencies: ``` ./build/sbt "sql/testOnly org.apache.spark.sql.execution.command.v2.ShowTablesSuite -- -z 'show table extended as json returns relation catalog table and view types'" ./build/mvn -pl sql/core -am -DskipTests -DskipJavaTests -DskipScalaTests -DskipMima -DskipCheckstyle test-compile MAVEN_MIRROR_URL=https://repo.maven.apache.org/maven2 ./build/mvn -pl sql/core -am -DskipTests -DskipJavaTests -DskipScalaTests -DskipMima -DskipCheckstyle test-compile ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) Closes #57362 from linhongliu-db/task/spark-53840-show-table-extended-json-v2-metadata/implementation. Authored-by: Linhong Liu <linhong.liu@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit e6df455) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This follow-up updates
SHOW TABLE EXTENDED ... AS JSONfor V2 catalogs to use catalog-provided summary metadata when producing extended JSON rows.RelationCatalog#listRelationSummaries, so relations such as views are preserved.TableCatalog#listTableSummaries.typefield now comes fromTableSummary.tableType()instead of being hard-coded toTABLE.Why are the changes needed?
SHOW TABLE EXTENDEDtext output uses V2 summary metadata for table type information, but the JSON path was falling back to identifier-only table listing and hard-coding the type asTABLE. That loses V2 table type metadata, and for relation catalogs it can also miss V2 views that are available through relation summaries.Does this PR introduce any user-facing change?
Yes. On unreleased Spark master,
SHOW TABLE EXTENDED ... AS JSONfor V2 catalogs now reports the catalog-provided table or view type metadata and preserves V2 relation-catalog views in the JSON output.How was this patch tested?
Added regression coverage in
ShowTablesSuite.Also checked:
GitHub PR checks passed on head
39f507910201d5dd08bbdb58125e1f7c0ac3809a:BuildNotify test workflowAttempted local Spark SQL suite validation, but the environment could not download required build dependencies:
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)