[SPARK-56611][SQL] Fix ALTER TABLE RENAME TO with catalog-qualified and table-only targets by lunar-shadow · Pull Request #55547 · apache/spark

lunar-shadow · 2026-04-24T19:58:43Z

What changes were proposed in this pull request?

This fixes ALTER TABLE ... RENAME TO in V2 catalogs for two cases that weren't handled correctly:

When the target name is catalog-qualified
When the target is just a table name (no namespace)

Changes:

Strip the catalog prefix from the target when it matches the source catalog
Inherit source namespace when target is table-only
Add tests covering the above cases

Why are the changes needed?

Right now, rename behaves incorrectly in a few scenarios.

For example:

ALTER TABLE testcat.ns.t1 RENAME TO testcat.ns.t2;

fails because asIdentifier treats everything except the last part as the namespace. That means the catalog name (testcat) incorrectly ends up inside the namespace:

Identifier.of(["testcat", "ns"], "t2")

instead of:

Identifier.of(["ns"], "t2")

Similarly:

ALTER TABLE testcat.ns.t1 RENAME TO t2;

creates an identifier with an empty namespace, instead of inheriting ns from the source table.

Steps to reproduce:

CREATE TABLE testcat.ns.t1 USING foo AS SELECT 1 AS id;
ALTER TABLE testcat.ns.t1 RENAME TO testcat.ns.t2;

This fails with a namespace resolution error due to the catalog being treated as part of the namespace.

Does this PR introduce any user-facing change?

Yes.

Catalog-qualified renames like ALTER TABLE catalog.ns.t1 RENAME TO catalog.ns.t2 now work correctly
Renaming to a table-only name (e.g., RENAME TO t2) now correctly keeps the original namespace
Cross-namespace renames (e.g., RENAME TO other_ns.t2) continue to work as before

How was this patch tested?

Added tests in DataSourceV2SQLSuite covering catalog-qualified renames, table-only renames, and cross-namespace renames.

Was this patch authored or co-authored using generative AI tooling?

No

…nd table-only targets

dongjoon-hyun · 2026-04-27T15:37:54Z

+
+      // Strip catalog prefix if the identifier is catalog-qualified.
+      val newNameParts =
+        if (rawNewNameParts.length > 1 && rawNewNameParts.head == catalog.name()) {


This looks like case-sensitive for catalog names which is incorrect, doesn't it? Please add a unit test case like the following?

ALTER TABLE testcat.ns.t1 RENAME TO TesTcaT.ns.t1_renamed

Thanks @dongjoon-hyun for catching this.

Fixed, now using SQLConf.get.resolver for case-insensitive comparison. Added a test case: ALTER TABLE testcat.ns.t1_renamed RENAME TO TesTcaT.ns.t1

dongjoon-hyun · 2026-04-27T15:50:34Z

+        }
+
+      val namespace =
+        if (newNameParts.length == 1) {


Please follow the community Scala coding style for val namespace = if (...) {.

https://spark.apache.org/contributing.html

https://github.com/databricks/scala-style-guide

val ret = if (condition) { Seq(1, 2, 3) } else { Seq(1, 2, 3) }

Fixed, thanks.

dongjoon-hyun · 2026-04-27T15:52:16Z

+    }
+  }
+
+  test("SPARK-56611: rename table across namespace should work for V2 catalogs") {


We don't need to mention for V2 catalogs phrase because this is inside DataSourceV2SQLSuite.

Done, renamed to "rename table across namespaces"

dongjoon-hyun · 2026-04-27T15:57:03Z

+        if (newNameParts.length == 1) {
+          oldIdent.namespace()
+        } else {
+          newNameParts.dropRight(1).toArray


In this case, we can use init.toArray like asIdentifier.

spark/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Implicits.scala

Line 210 in e51baae

def asIdentifier: Identifier = Identifier.of(parts.init.toArray, parts.last)

- newNameParts.dropRight(1).toArray + newNameParts.init.toArray

Fixed, thanks.

dongjoon-hyun · 2026-04-27T16:03:33Z

+      sql("CREATE TABLE testcat.ns1.t1 USING foo AS SELECT id, data FROM source")
+      checkAnswer(sql("SHOW TABLES FROM testcat.ns1"), Seq(Row("ns1", "t1", false)))
+
+      sql("ALTER TABLE testcat.ns1.t1 RENAME TO ns2.t1")


Dose this PR aim to support ALTER TABLE testcat.ns1.t1 RENAME TO othercat.ns2.t1? Could you add a test case?

Cross-catalog rename is not addressed as part of this PR. I noticed it currently silently treats the other catalog name as a namespace. I'll file a follow-up JIRA for proper validation.

Shouldn't the above be interpreted as move (rename) t1 table from ns1 namespace to othercat.ns2 namespace in testcat catalog? So doesn't the current PR work as expected?

Right, cross-catalog rename may not be a valid operation as RENAME operates within a single catalog. So othercat.ns2.t1 is correctly treated as a namespace path within testcat. Added a test case to verify this. No follow-up needed, thanks for the clarification.

dongjoon-hyun · 2026-04-27T16:08:06Z

+        if (rawNewNameParts.length > 1 && rawNewNameParts.head == catalog.name()) {
+          rawNewNameParts.tail
+        } else {
+          rawNewNameParts


Do we have a test case for this?

Yes, both RENAME TO testcat.ns.t1_renamed and RENAME TO TesTcaT.ns.t1 in the test hit this code path.

dongjoon-hyun · 2026-04-27T16:10:55Z

cc @peter-toth , too.

…s review feedback

peter-toth · 2026-04-28T10:13:39Z

+      val namespace = if (newNameParts.length == 1) {
+        oldIdent.namespace()
+      } else {
+        newNameParts.init.toArray
+      }
+
+      val newIdent = Identifier.of(namespace, newNameParts.last)


Minor nit:

Suggested change

val namespace = if (newNameParts.length == 1) {

oldIdent.namespace()

} else {

newNameParts.init.toArray

}

val newIdent = Identifier.of(namespace, newNameParts.last)

val newIdent = if (newNameParts.length == 1) {

Identifier.of(oldIdent.namespace(), newNameParts.last)

} else {

newNameParts.asIdentifier

}

adopted this change, thanks!

peter-toth · 2026-04-28T10:17:12Z

+      sql("CREATE TABLE testcat.ns1.t1 USING foo AS SELECT id, data FROM source")
+      checkAnswer(sql("SHOW TABLES FROM testcat.ns1"), Seq(Row("ns1", "t1", false)))
+
+      sql("ALTER TABLE testcat.ns1.t1 RENAME TO ns2.t1")


Shouldn't the above be interpreted as move (rename) t1 table from ns1 namespace to othercat.ns2 namespace in testcat catalog? So doesn't the current PR work as expected?

peter-toth · 2026-04-28T10:19:42Z

cc @cloud-fan, @gengliangwang

…alog namespace resolution

lunar-shadow force-pushed the fix/rename-table-with-catalog branch from 669a517 to 8163d49 Compare April 24, 2026 20:04

[SPARK-56611][SQL] Fix ALTER TABLE RENAME TO with catalog-qualified a…

cf3a248

…nd table-only targets

lunar-shadow force-pushed the fix/rename-table-with-catalog branch from 8163d49 to cf3a248 Compare April 25, 2026 05:00

dongjoon-hyun requested changes Apr 27, 2026

View reviewed changes

[SPARK-56611][SQL] Fix case-insensitive catalog comparison and addres…

c57ac4b

…s review feedback

peter-toth approved these changes Apr 28, 2026

View reviewed changes

[SPARK-56611][SQL] Adopt review suggestion and add test for cross-cat…

6158669

…alog namespace resolution

Conversation

lunar-shadow commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Apr 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peter-toth commented Apr 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lunar-shadow commented Apr 24, 2026 •

edited

Loading