[SPARK-35105][SQL] Support multiple paths for ADD FILE/JAR/ARCHIVE commands

[sarutak]

What changes were proposed in this pull request?

This PR extends ADD FILE/JAR/ARCHIVE commands to be able to take multiple path arguments like Hive.

Why are the changes needed?

To make those commands more useful.

Does this PR introduce any user-facing change?

Yes. In the current implementation, those commands can take a path which contains whitespaces without enclose it by neither ' nor " but after this change, users need to enclose such paths.
I've note this incompatibility in the migration guide.

How was this patch tested?

New tests.

[github-actions]

Test build #755422826 for PR 32205 at commit 7cd0628.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42061/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42061/

[SparkQA]

Test build #137486 has finished for PR 32205 at commit 7cd0628.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

@sarutak would you mind rebasing your branch against the latest master branch in Apache Spark? there have been a couple of CI updates so it has to be synced.

[sarutak]

@HyukjinKwon Thank you for letting me know. I'll update it.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42113/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42113/

[SparkQA]

Test build #137540 has finished for PR 32205 at commit 002eaed.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• class PandasOnSparkFrameMethods(object):
• class PandasOnSparkSeriesMethods(object):
• class PandasOnSparkPlotAccessor(PandasObject):
• class PandasOnSparkBarPlot(PandasBarPlot, TopNPlotBase):
• class PandasOnSparkBoxPlot(PandasBoxPlot, BoxPlotBase):
• class PandasOnSparkHistPlot(PandasHistPlot, HistogramPlotBase):
• class PandasOnSparkPiePlot(PandasPiePlot, TopNPlotBase):
• class PandasOnSparkAreaPlot(PandasAreaPlot, SampledPlotBase):
• class PandasOnSparkLinePlot(PandasLinePlot, SampledPlotBase):
• class PandasOnSparkBarhPlot(PandasBarhPlot, TopNPlotBase):
• class PandasOnSparkScatterPlot(PandasScatterPlot, TopNPlotBase):
• class PandasOnSparkKdePlot(PandasKdePlot, KdePlotBase):
• class PandasOnSparkUsageLogger(object):

[SparkQA]

Test build #137572 has finished for PR 32205 at commit 7cd0628.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42132/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42132/

[SparkQA]

Test build #137596 has finished for PR 32205 at commit 9fe3eee.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[sarutak]

@HyukjinKwon Do you have any comment on this change?

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42528/

[SparkQA]

Test build #138009 has finished for PR 32205 at commit c7175de.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

Just to clarify, these are all supported in Hive too, right? If that's the case, I think the change is fine

[sarutak]

@HyukjinKwon

Just to clarify, these are all supported in Hive too, right? If that's the case, I think the change is fine

Yes. It works in Hive as documented (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Commands).
I also confirmed with Hive 2.3.7.

[sarutak]

Merged to master. Thanks @HyukjinKwon for the review.