[SPARK-33364][SQL][FOLLOWUP] Refine the catalog v2 API to purge a table

[cloud-fan]

What changes were proposed in this pull request?

This is a followup of #30267

Inspired by #30886, it's better to have 2 methods def dropTable and def purgeTable, than def dropTable(ident) and def dropTable(ident, purge).

Why are the changes needed?

1. make the APIs orthogonal. Previously, def dropTable(ident, purge) calls def dropTable(ident) and is a superset.
2. simplifies the catalog implementation a little bit. Now the if (purge) ... else ... check is done at the Spark side.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

existing tests

[cloud-fan]

cc @imback82 @rdblue @MaxGekk

[MaxGekk]

Are you going to backport this to branch-3.1?

[cloud-fan]

Yea, otherwise it's a breaking change and is not accepted.

[MaxGekk]

There are tests for the dropTable() method in TableCatalogSuite:

spark/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/TableCatalogSuite.scala

Lines 620 to 633 in 4e5d2e0

	test("dropTable") {
	val catalog = newCatalog()
	assert(!catalog.tableExists(testIdent))
	catalog.createTable(testIdent, schema, Array.empty, emptyProps)
	assert(catalog.tableExists(testIdent))
	val wasDropped = catalog.dropTable(testIdent)
	assert(wasDropped)
	assert(!catalog.tableExists(testIdent))
	}

Should we add similar primitive tests for purgeTable() there?

[MaxGekk]

LGTM

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37820/

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37826/

[imback82]

LGTM

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37826/

[SparkQA]

Test build #133223 has finished for PR 30890 at commit 879ea6d.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #133228 has finished for PR 30890 at commit 3b57205.

• This patch fails SparkR unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

LGTM

[HyukjinKwon]

Merged to master and branch-3.1.