Skip to content

[SPARK-56611][SQL] Fix ALTER TABLE RENAME TO with catalog-qualified and table-only targets #55547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lunar-shadow wants to merge 3 commits into
base: master
Choose a base branch
from
Open

[SPARK-56611][SQL] Fix ALTER TABLE RENAME TO with catalog-qualified and table-only targets #55547

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cf3a248
[SPARK-56611][SQL] Fix ALTER TABLE RENAME TO with catalog-qualified a…
lunar-shadow Apr 24, 2026
c57ac4b
[SPARK-56611][SQL] Fix case-insensitive catalog comparison and addres…
lunar-shadow Apr 28, 2026
6158669
[SPARK-56611][SQL] Adopt review suggestion and add test for cross-cat…
lunar-shadow Apr 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions ...e/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -448,14 +448,29 @@ class DataSourceV2Strategy(session: SparkSession) extends Strategy with Predicat
case _: NoopCommand =>
LocalTableScanExec(Nil, Nil, None) :: Nil

case RenameTable(r @ ResolvedTable(catalog, oldIdent, _, _), newIdent, isView) =>
case RenameTable(r @ ResolvedTable(catalog, oldIdent, _, _), rawNewNameParts, isView) =>
if (isView) {
throw QueryCompilationErrors.cannotRenameTableWithAlterViewError()
}

// Strip catalog prefix if the identifier is catalog-qualified.
val newNameParts = if (rawNewNameParts.length > 1 &&
SQLConf.get.resolver(rawNewNameParts.head, catalog.name())) {
rawNewNameParts.tail
} else {
rawNewNameParts
}

val newIdent = if (newNameParts.length == 1) {
Identifier.of(oldIdent.namespace(), newNameParts.last)
} else {
newNameParts.asIdentifier
}

RenameTableExec(
catalog,
oldIdent,
newIdent.asIdentifier,
newIdent,
invalidateTableCache(r),
session.sharedState.cacheManager.cacheQuery) :: Nil

Expand Down
47 changes: 47 additions & 0 deletions sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2486,6 +2486,53 @@ class DataSourceV2SQLSuiteV1Filter
ExpectedContext("testcat.ns.tbl", 11, 10 + "testcat.ns.tbl".length))
}

test("SPARK-56611: rename table with catalog-qualified identifier") {
withTable("testcat.ns.t1", "testcat.ns.t1_renamed") {
sql("CREATE TABLE testcat.ns.t1 USING foo AS SELECT id, data FROM source")
checkAnswer(sql("SHOW TABLES FROM testcat.ns"), Seq(Row("ns", "t1", false)))

// rename with just table name, inherits namespace from source
sql("ALTER TABLE testcat.ns.t1 RENAME TO t1_renamed")
checkAnswer(sql("SHOW TABLES FROM testcat.ns"), Seq(Row("ns", "t1_renamed", false)))

// rename with namespace-qualified name
sql("ALTER TABLE testcat.ns.t1_renamed RENAME TO ns.t1")
checkAnswer(sql("SHOW TABLES FROM testcat.ns"), Seq(Row("ns", "t1", false)))

// rename with catalog-qualified target identifier
sql("ALTER TABLE testcat.ns.t1 RENAME TO testcat.ns.t1_renamed")
checkAnswer(sql("SHOW TABLES FROM testcat.ns"), Seq(Row("ns", "t1_renamed", false)))

// rename with case-insensitive catalog-qualified target identifier
sql("ALTER TABLE testcat.ns.t1_renamed RENAME TO TesTcaT.ns.t1")
checkAnswer(sql("SHOW TABLES FROM testcat.ns"), Seq(Row("ns", "t1", false)))
}
}

test("SPARK-56611: rename table across namespaces") {
withTable("testcat.ns1.t1", "testcat.ns2.t1") {
sql("CREATE TABLE testcat.ns1.t1 USING foo AS SELECT id, data FROM source")
checkAnswer(sql("SHOW TABLES FROM testcat.ns1"), Seq(Row("ns1", "t1", false)))

sql("ALTER TABLE testcat.ns1.t1 RENAME TO ns2.t1")
Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dose this PR aim to support ALTER TABLE testcat.ns1.t1 RENAME TO othercat.ns2.t1? Could you add a test case?

Copy link
Copy Markdown
Author

@lunar-shadow lunar-shadow Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-catalog rename is not addressed as part of this PR. I noticed it currently silently treats the other catalog name as a namespace. I'll file a follow-up JIRA for proper validation.

Copy link
Copy Markdown
Contributor

@peter-toth peter-toth Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the above be interpreted as move (rename) t1 table from ns1 namespace to othercat.ns2 namespace in testcat catalog? So doesn't the current PR work as expected?

Copy link
Copy Markdown
Author

@lunar-shadow lunar-shadow Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, cross-catalog rename may not be a valid operation as RENAME operates within a single catalog. So othercat.ns2.t1 is correctly treated as a namespace path within testcat. Added a test case to verify this. No follow-up needed, thanks for the clarification.

checkAnswer(sql("SHOW TABLES FROM testcat.ns1"), Nil)
checkAnswer(sql("SHOW TABLES FROM testcat.ns2"), Seq(Row("ns2", "t1", false)))
}
}

test("SPARK-56611: rename target is resolved within source catalog") {
withTable("testcat.ns1.t1", "testcat.testcat2.ns2.t1") {
sql("CREATE TABLE testcat.ns1.t1 USING foo AS SELECT id, data FROM source")

// testcat2 is a valid catalog name, but in the rename target it is
// treated as a namespace within testcat (the source table's catalog).
sql("ALTER TABLE testcat.ns1.t1 RENAME TO testcat2.ns2.t1")
checkAnswer(sql("SHOW TABLES FROM testcat.ns1"), Nil)
checkAnswer(
sql("SHOW TABLES FROM testcat.testcat2.ns2"), Seq(Row("testcat2.ns2", "t1", false)))
}
}

test("ANALYZE TABLE") {
val t = "testcat.ns1.ns2.tbl"
withTable(t) {
Expand Down