Skip to content

[SPARK-53840][SQL][FollowUp] Make SHOW TABLE EXTENDED AS JSON match V2 metadata#57362

Closed
linhongliu-db wants to merge 2 commits into
apache:masterfrom
linhongliu-db:task/spark-53840-show-table-extended-json-v2-metadata/implementation
Closed

[SPARK-53840][SQL][FollowUp] Make SHOW TABLE EXTENDED AS JSON match V2 metadata#57362
linhongliu-db wants to merge 2 commits into
apache:masterfrom
linhongliu-db:task/spark-53840-show-table-extended-json-v2-metadata/implementation

Conversation

@linhongliu-db

@linhongliu-db linhongliu-db commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This follow-up updates SHOW TABLE EXTENDED ... AS JSON for V2 catalogs to use catalog-provided summary metadata when producing extended JSON rows.

  • For V2 relation catalogs, extended JSON now uses RelationCatalog#listRelationSummaries, so relations such as views are preserved.
  • For V2 table catalogs, extended JSON now uses TableCatalog#listTableSummaries.
  • The extended JSON type field now comes from TableSummary.tableType() instead of being hard-coded to TABLE.
  • V2 command-suite coverage now checks the V2 summary table type and covers a relation catalog returning both a table and a view.

Why are the changes needed?

SHOW TABLE EXTENDED text output uses V2 summary metadata for table type information, but the JSON path was falling back to identifier-only table listing and hard-coding the type as TABLE. That loses V2 table type metadata, and for relation catalogs it can also miss V2 views that are available through relation summaries.

Does this PR introduce any user-facing change?

Yes. On unreleased Spark master, SHOW TABLE EXTENDED ... AS JSON for V2 catalogs now reports the catalog-provided table or view type metadata and preserves V2 relation-catalog views in the JSON output.

How was this patch tested?

Added regression coverage in ShowTablesSuite.

Also checked:

git diff --check apache/master..HEAD

GitHub PR checks passed on head 39f507910201d5dd08bbdb58125e1f7c0ac3809a:

  • Build
  • Notify test workflow

Attempted local Spark SQL suite validation, but the environment could not download required build dependencies:

./build/sbt "sql/testOnly org.apache.spark.sql.execution.command.v2.ShowTablesSuite -- -z 'show table extended as json returns relation catalog table and view types'"
./build/mvn -pl sql/core -am -DskipTests -DskipJavaTests -DskipScalaTests -DskipMima -DskipCheckstyle test-compile
MAVEN_MIRROR_URL=https://repo.maven.apache.org/maven2 ./build/mvn -pl sql/core -am -DskipTests -DskipJavaTests -DskipScalaTests -DskipMima -DskipCheckstyle test-compile

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Codex (GPT-5)

@linhongliu-db linhongliu-db changed the title [FollowUp][SPARK-53840][SQL] Make SHOW TABLE EXTENDED AS JSON match V2 metadata [SPARK-53840][SQL][FollowUp] Make SHOW TABLE EXTENDED AS JSON match V2 metadata Jul 20, 2026
@linhongliu-db
linhongliu-db marked this pull request as ready for review July 20, 2026 18:10
@linhongliu-db
linhongliu-db force-pushed the task/spark-53840-show-table-extended-json-v2-metadata/implementation branch from 1e94281 to 57f4a03 Compare July 20, 2026 20:56

@uros-b uros-b left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @linhongliu-db! cc @cloud-fan @urosstan-db

@cloud-fan cloud-fan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 blocking, 0 non-blocking, 0 nits.
The implementation and regression coverage are consistent with the V2 catalog contracts; no actionable findings.

Verification

Traced both V2 catalog branches through listRelationSummaries / listTableSummaries, verified pattern filtering remains on relation names, compared the relation path with ShowTablesExec, and checked the TableCatalog and RelationCatalog default summary contracts. No tests were run as part of this review.

@cloud-fan cloud-fan closed this in e6df455 Jul 21, 2026
cloud-fan pushed a commit that referenced this pull request Jul 21, 2026
…2 metadata

### What changes were proposed in this pull request?

This follow-up updates `SHOW TABLE EXTENDED ... AS JSON` for V2 catalogs to use catalog-provided summary metadata when producing extended JSON rows.

- For V2 relation catalogs, extended JSON now uses `RelationCatalog#listRelationSummaries`, so relations such as views are preserved.
- For V2 table catalogs, extended JSON now uses `TableCatalog#listTableSummaries`.
- The extended JSON `type` field now comes from `TableSummary.tableType()` instead of being hard-coded to `TABLE`.
- V2 command-suite coverage now checks the V2 summary table type and covers a relation catalog returning both a table and a view.

### Why are the changes needed?

`SHOW TABLE EXTENDED` text output uses V2 summary metadata for table type information, but the JSON path was falling back to identifier-only table listing and hard-coding the type as `TABLE`. That loses V2 table type metadata, and for relation catalogs it can also miss V2 views that are available through relation summaries.

### Does this PR introduce _any_ user-facing change?

Yes. On unreleased Spark master, `SHOW TABLE EXTENDED ... AS JSON` for V2 catalogs now reports the catalog-provided table or view type metadata and preserves V2 relation-catalog views in the JSON output.

### How was this patch tested?

Added regression coverage in `ShowTablesSuite`.

Also checked:

```
git diff --check apache/master..HEAD
```

GitHub PR checks passed on head `39f507910201d5dd08bbdb58125e1f7c0ac3809a`:

- `Build`
- `Notify test workflow`

Attempted local Spark SQL suite validation, but the environment could not download required build dependencies:

```
./build/sbt "sql/testOnly org.apache.spark.sql.execution.command.v2.ShowTablesSuite -- -z 'show table extended as json returns relation catalog table and view types'"
./build/mvn -pl sql/core -am -DskipTests -DskipJavaTests -DskipScalaTests -DskipMima -DskipCheckstyle test-compile
MAVEN_MIRROR_URL=https://repo.maven.apache.org/maven2 ./build/mvn -pl sql/core -am -DskipTests -DskipJavaTests -DskipScalaTests -DskipMima -DskipCheckstyle test-compile
```

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Codex (GPT-5)

Closes #57362 from linhongliu-db/task/spark-53840-show-table-extended-json-v2-metadata/implementation.

Authored-by: Linhong Liu <linhong.liu@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(cherry picked from commit e6df455)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
@cloud-fan

Copy link
Copy Markdown
Contributor

Merge Summary:

Posted by merge_spark_pr.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants