[SPARK-55250][SQL][FOLLOWUP] Skip createNamespace for IF NOT EXISTS on existing namespace

[cloud-fan]

What changes were proposed in this pull request?

Follow-up to SPARK-55250. Add a recovery path to CreateNamespaceExec.run() so IF NOT EXISTS is a no-op when the namespace already exists, even if the catalog surfaces an error other than NamespaceAlreadyExistsException:

try {
  val ownership = Map(PROP_OWNER -> Utils.getCurrentUserName())
  catalog.createNamespace(ns, (properties ++ ownership).asJava)
} catch {
  case _: NamespaceAlreadyExistsException if ifNotExists =>
    logWarning(...)
  case NonFatal(e) if ifNotExists =>
    val exists = try catalog.namespaceExists(ns) catch { case NonFatal(_) => false }
    if (exists) logWarning(..., e) else throw e
}

The unconditional createNamespace call introduced by SPARK-55250 is preserved as the first step, so the perf win from that PR is kept on every happy path. The namespaceExists fallback runs only when createNamespace has already failed — a path that was previously an unrecoverable error.

Why are the changes needed?

SPARK-55250 changed CREATE NAMESPACE IF NOT EXISTS foo from "check existence first, skip if present" to "always call createNamespace, catch NamespaceAlreadyExistsException". This relies on the catalog raising NamespaceAlreadyExistsException rather than some other error when the namespace is pre-existing.

For SupportsNamespaces implementations that validate the request (ACLs, properties, etc.) before checking existence, this assumption doesn't hold: the validation error surfaces first, the NamespaceAlreadyExistsException is never thrown, and the IF NOT EXISTS no-op semantic is lost.

The fix asks the only question that actually matters under IF NOT EXISTS after a failure: "does the namespace exist now?" If yes, intent satisfied; otherwise the original error propagates.

Does this PR introduce any user-facing change?

Yes. CREATE NAMESPACE IF NOT EXISTS foo is again a no-op when foo already exists, regardless of which error the catalog raised on the create attempt. This matches the pre-SPARK-55250 contract.

RPC accounting (using UC as an example of a catalog that validates before existence check):

Scenario	SPARK-55250	This PR
no IF NOT EXISTS	1	1
IF NOT EXISTS, foo absent	1	1
IF NOT EXISTS, foo exists, create succeeds-or-throws-AlreadyExists	1	1
IF NOT EXISTS, foo exists, create throws something else	1 (surfaces error ❌)	2 (recovers ✅)

How was this patch tested?

• New ValidatingInMemoryTableCatalog (an InMemoryTableCatalog subclass that validates before checking existence, so a pre-existing namespace raises a non-NamespaceAlreadyExistsException) registered as validating_test_catalog in v2.CommandSuiteBase.
• New SQL-level regression test in v2.CreateNamespaceSuite that creates a namespace and re-runs CREATE NAMESPACE IF NOT EXISTS against that catalog — fails on master, passes with this PR.
• Existing org.apache.spark.sql.hive.execution.command.CreateNamespaceSuite "hive client calls" test still asserts exactly 1 RPC for each of the three CREATE NAMESPACE shapes it covers — confirming SPARK-55250's perf win is preserved on the happy path.
• Existing v1 / v2 CreateNamespaceSuite pass.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.7)

[cloud-fan]

thanks for review, merging to master/4.x/4.2!