[SPARK-37546][SQL] V2 ReplaceTableAsSelect command should qualify location

[huaxingao]

What changes were proposed in this pull request?

Currently, v2 RTAS command doesn't qualify the location:

spark.sql("REPLACE TABLE testcat.t USING foo LOCATION '/tmp/foo' AS SELECT id FROM source")
spark.sql("DESCRIBE EXTENDED testcat.t").filter("col_name = 'Location'").show

+--------+-------------+-------+
|col_name|    data_type|comment|
+--------+-------------+-------+
|Location|/tmp/foo     |       |
+--------+-------------+-------+

, whereas v1 command qualifies the location as file:/tmp/foo which is the correct behavior since the default filesystem can change for different sessions.

Why are the changes needed?

This PR proposes to store the qualified location in order to prevent the issue where default filesystem changes for different sessions.

Does this PR introduce any user-facing change?

Yes, now, v2 RTAS command will store qualified location:

+--------+-------------+-------+
|col_name|    data_type|comment|
+--------+-------------+-------+
|Location|file:/tmp/foo|       |
+--------+-------------+-------+

How was this patch tested?

new test

[huaxingao]

cc @imback82

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50399/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50399/

[imback82]

LGTM if tests pass. Thanks @huaxingao!

[SparkQA]

Test build #145923 has finished for PR 34807 at commit 19b656a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50402/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50402/

[SparkQA]

Test build #145926 has finished for PR 34807 at commit b065e18.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[huaxingao]

@cloud-fan

[cloud-fan]

can we add a new test case? In the future I believe we will have separate test suites for CREATE and REPLACE TABLE and we need separate test cases anyway.

[cloud-fan]

shall we add a new method qualifyLocInTableSpec(tableSpec): TableSpec to simplify the code here and other places?

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50419/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50419/

[cloud-fan]

thanks, merging to master!

[SparkQA]

Test build #145944 has finished for PR 34807 at commit e1104a0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #145943 has finished for PR 34807 at commit 5ff0c26.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[huaxingao]

Thanks! @cloud-fan @imback82