[SPARK-20590][SQL] Use Spark internal datasource if multiples are found for the same shorten name #17916

HyukjinKwon · 2017-05-09T07:43:51Z

What changes were proposed in this pull request?

One of the common usability problems around reading data in spark (particularly CSV) is that there can often be a conflict between different readers in the classpath.

As an example, if someone launches a 2.x spark shell with the spark-csv package in the classpath, Spark currently fails in an extremely unfriendly way (see databricks/spark-csv#367):

./bin/spark-shell --packages com.databricks:spark-csv_2.11:1.5.0
scala> val df = spark.read.csv("/foo/bar.csv")
java.lang.RuntimeException: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), please specify the fully qualified class name.
  at scala.sys.package$.error(package.scala:27)
  at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:574)
  at org.apache.spark.sql.execution.datasources.DataSource.providingClass$lzycompute(DataSource.scala:85)
  at org.apache.spark.sql.execution.datasources.DataSource.providingClass(DataSource.scala:85)
  at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:295)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178)
  at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:533)
  at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:412)
  ... 48 elided

This PR proposes a simple way of fixing this error by picking up the internal datasource if there is single (the datasource that has "org.apache.spark" prefix).

scala> spark.range(1).write.format("csv").mode("overwrite").save("/tmp/abc")
17/05/10 09:47:44 WARN DataSource: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat,
com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat).

scala> spark.range(1).write.format("Csv").mode("overwrite").save("/tmp/abc")
17/05/10 09:47:52 WARN DataSource: Multiple sources found for Csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat,
com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat).

How was this patch tested?

Manually tested as below:

./bin/spark-shell --packages com.databricks:spark-csv_2.11:1.5.0

spark.sparkContext.setLogLevel("WARN")

positive cases:

scala> spark.range(1).write.format("csv").mode("overwrite").save("/tmp/abc")
17/05/10 09:47:44 WARN DataSource: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat,
com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat).

scala> spark.range(1).write.format("Csv").mode("overwrite").save("/tmp/abc")
17/05/10 09:47:52 WARN DataSource: Multiple sources found for Csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat,
com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat).

(newlines were inserted for readability).

scala> spark.range(1).write.format("com.databricks.spark.csv").mode("overwrite").save("/tmp/abc")

scala> spark.range(1).write.format("org.apache.spark.sql.execution.datasources.csv.CSVFileFormat").mode("overwrite").save("/tmp/abc")

negative cases:

scala> spark.range(1).write.format("com.databricks.spark.csv.CsvRelation").save("/tmp/abc")
java.lang.InstantiationException: com.databricks.spark.csv.CsvRelation
...

scala> spark.range(1).write.format("com.databricks.spark.csv.CsvRelatio").save("/tmp/abc")
java.lang.ClassNotFoundException: Failed to find data source: com.databricks.spark.csv.CsvRelatio. Please find packages at http://spark.apache.org/third-party-projects.html
...

HyukjinKwon · 2017-05-09T07:44:56Z

cc @sameeragarwal and @cloud-fan, I just came up with another way and I opened this to show my idea. What do you think about this?

SparkQA · 2017-05-09T09:14:06Z

Test build #76658 has finished for PR 17916 at commit 03dc0f6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2017-05-09T12:21:29Z

sql/core/src/test/scala/org/apache/spark/sql/sources/DDLSourceLoadSuite.scala



 // please note that the META-INF/services had to be modified for the test directory for this to work
 class DDLSourceLoadSuite extends DataSourceTest with SharedSQLContext {

  test("data sources with the same name") {
-    intercept[RuntimeException] {
-      spark.read.format("Fluet da Bomb").load()


we still need a test case to cover the conflicting data source case.

…al datasources and internal datasources

SparkQA · 2017-05-09T14:02:39Z

Test build #76669 has finished for PR 17916 at commit 2dce84c.

This patch fails from timeout after a configured wait of `250m`.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class FakeSourceTwo extends RelationProvider with DataSourceRegister

SparkQA · 2017-05-09T17:12:41Z

Test build #76681 has finished for PR 17916 at commit 741c913.

This patch fails from timeout after a configured wait of `250m`.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class FakeSourceFour extends RelationProvider with DataSourceRegister
class FakeExternalSourceOne extends RelationProvider with DataSourceRegister
class FakeExternalSourceTwo extends RelationProvider with DataSourceRegister
class FakeExternalSourceThree extends RelationProvider with DataSourceRegister

SparkQA · 2017-05-09T17:27:39Z

Test build #76682 has finished for PR 17916 at commit 8c40eab.

This patch fails from timeout after a configured wait of `250m`.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class FakeExternalSourceOne extends RelationProvider with DataSourceRegister
class FakeExternalSourceTwo extends RelationProvider with DataSourceRegister
class FakeExternalSourceThree extends RelationProvider with DataSourceRegister

HyukjinKwon · 2017-05-09T20:20:19Z

retest this please

SparkQA · 2017-05-09T22:35:13Z

Test build #76696 has finished for PR 17916 at commit 8c40eab.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class FakeExternalSourceOne extends RelationProvider with DataSourceRegister
class FakeExternalSourceTwo extends RelationProvider with DataSourceRegister
class FakeExternalSourceThree extends RelationProvider with DataSourceRegister

sameeragarwal · 2017-05-09T22:36:20Z

Thanks @HyukjinKwon, I like this approach better!

One limitation of this patch however is that if there are ever two internal datasources in Spark with the same shortName, we might've introduced some inadvertent randomness here (by picking the first datasource from the sequence). Thoughts?

HyukjinKwon · 2017-05-09T22:54:42Z

Yea. Probably, I think it should check if the length is single with another test as well and checking this would not harm.

HyukjinKwon · 2017-05-09T23:23:28Z

sql/core/src/test/scala/org/apache/spark/sql/sources/DDLSourceLoadSuite.scala

+    assert(e.getMessage.contains("Multiple sources found for Fluet da Bomb"))
+  }
+
+  test("data sources with the same name - internal data source/external data source") {


So, we will only allow this case.

sameeragarwal · 2017-05-09T23:35:19Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala

+          val internalSources = sources.filter(_.getClass.getName.startsWith("org.apache.spark"))
+          if (internalSources.size == 1) {
+            logWarning(s"Multiple sources found for $provider1 (${sourceNames.mkString(", ")}), " +
+              "please specify the fully qualified class name. " +


nit: this isn't really actionable so we can consider deleting it from here and say something like "defaulting to the internal ..."

sameeragarwal · 2017-05-09T23:35:35Z

LGTM

HyukjinKwon · 2017-05-09T23:38:50Z

Thanks for approving this approach. I will handle the comment soon.

cloud-fan · 2017-05-10T00:46:06Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala

+              s"Using the internal datasource (${internalSources.head.getClass.getName}).")
+            internalSources.head.getClass
+          } else {
+            sys.error(s"Multiple sources found for $provider1 (${sourceNames.mkString(", ")}), " +


nit: let's throw analysis exception

cloud-fan · 2017-05-10T00:47:03Z

LGTM

SparkQA · 2017-05-10T01:25:21Z

Test build #76709 has finished for PR 17916 at commit 4450da7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2017-05-10T01:40:41Z

LGTM, pending jenkins

viirya · 2017-05-10T01:52:02Z

@HyukjinKwon Shall we also update the PR description?

HyukjinKwon · 2017-05-10T01:59:20Z

Sure.

viirya · 2017-05-10T02:09:07Z

LGTM

SparkQA · 2017-05-10T03:08:33Z

Test build #76714 has finished for PR 17916 at commit 7a464ad.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-05-10T03:25:14Z

Test build #76715 has finished for PR 17916 at commit 96cf1a9.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
s\"($

…nd for the same shorten name ## What changes were proposed in this pull request? One of the common usability problems around reading data in spark (particularly CSV) is that there can often be a conflict between different readers in the classpath. As an example, if someone launches a 2.x spark shell with the spark-csv package in the classpath, Spark currently fails in an extremely unfriendly way (see databricks/spark-csv#367): ```bash ./bin/spark-shell --packages com.databricks:spark-csv_2.11:1.5.0 scala> val df = spark.read.csv("/foo/bar.csv") java.lang.RuntimeException: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), please specify the fully qualified class name. at scala.sys.package$.error(package.scala:27) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:574) at org.apache.spark.sql.execution.datasources.DataSource.providingClass$lzycompute(DataSource.scala:85) at org.apache.spark.sql.execution.datasources.DataSource.providingClass(DataSource.scala:85) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:295) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178) at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:533) at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:412) ... 48 elided ``` This PR proposes a simple way of fixing this error by picking up the internal datasource if there is single (the datasource that has "org.apache.spark" prefix). ```scala scala> spark.range(1).write.format("csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:44 WARN DataSource: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` ```scala scala> spark.range(1).write.format("Csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:52 WARN DataSource: Multiple sources found for Csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` ## How was this patch tested? Manually tested as below: ```bash ./bin/spark-shell --packages com.databricks:spark-csv_2.11:1.5.0 ``` ```scala spark.sparkContext.setLogLevel("WARN") ``` **positive cases**: ```scala scala> spark.range(1).write.format("csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:44 WARN DataSource: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` ```scala scala> spark.range(1).write.format("Csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:52 WARN DataSource: Multiple sources found for Csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` (newlines were inserted for readability). ```scala scala> spark.range(1).write.format("com.databricks.spark.csv").mode("overwrite").save("/tmp/abc") ``` ```scala scala> spark.range(1).write.format("org.apache.spark.sql.execution.datasources.csv.CSVFileFormat").mode("overwrite").save("/tmp/abc") ``` **negative cases**: ```scala scala> spark.range(1).write.format("com.databricks.spark.csv.CsvRelation").save("/tmp/abc") java.lang.InstantiationException: com.databricks.spark.csv.CsvRelation ... ``` ```scala scala> spark.range(1).write.format("com.databricks.spark.csv.CsvRelatio").save("/tmp/abc") java.lang.ClassNotFoundException: Failed to find data source: com.databricks.spark.csv.CsvRelatio. Please find packages at http://spark.apache.org/third-party-projects.html ... ``` Author: hyukjinkwon <gurwls223@gmail.com> Closes #17916 from HyukjinKwon/datasource-detect. (cherry picked from commit 3d2131a) Signed-off-by: Wenchen Fan <wenchen@databricks.com>

cloud-fan · 2017-05-10T05:47:27Z

thanks, merging to master/2.2!

HyukjinKwon · 2017-05-10T05:48:53Z

Thanks everyone.

chrishfish · 2017-05-16T20:01:58Z

Awesome @HyukjinKwon glad this issue has been resolved permanently 👍

…nd for the same shorten name ## What changes were proposed in this pull request? One of the common usability problems around reading data in spark (particularly CSV) is that there can often be a conflict between different readers in the classpath. As an example, if someone launches a 2.x spark shell with the spark-csv package in the classpath, Spark currently fails in an extremely unfriendly way (see databricks/spark-csv#367): ```bash ./bin/spark-shell --packages com.databricks:spark-csv_2.11:1.5.0 scala> val df = spark.read.csv("/foo/bar.csv") java.lang.RuntimeException: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), please specify the fully qualified class name. at scala.sys.package$.error(package.scala:27) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:574) at org.apache.spark.sql.execution.datasources.DataSource.providingClass$lzycompute(DataSource.scala:85) at org.apache.spark.sql.execution.datasources.DataSource.providingClass(DataSource.scala:85) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:295) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178) at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:533) at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:412) ... 48 elided ``` This PR proposes a simple way of fixing this error by picking up the internal datasource if there is single (the datasource that has "org.apache.spark" prefix). ```scala scala> spark.range(1).write.format("csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:44 WARN DataSource: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` ```scala scala> spark.range(1).write.format("Csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:52 WARN DataSource: Multiple sources found for Csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` ## How was this patch tested? Manually tested as below: ```bash ./bin/spark-shell --packages com.databricks:spark-csv_2.11:1.5.0 ``` ```scala spark.sparkContext.setLogLevel("WARN") ``` **positive cases**: ```scala scala> spark.range(1).write.format("csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:44 WARN DataSource: Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` ```scala scala> spark.range(1).write.format("Csv").mode("overwrite").save("/tmp/abc") 17/05/10 09:47:52 WARN DataSource: Multiple sources found for Csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), defaulting to the internal datasource (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat). ``` (newlines were inserted for readability). ```scala scala> spark.range(1).write.format("com.databricks.spark.csv").mode("overwrite").save("/tmp/abc") ``` ```scala scala> spark.range(1).write.format("org.apache.spark.sql.execution.datasources.csv.CSVFileFormat").mode("overwrite").save("/tmp/abc") ``` **negative cases**: ```scala scala> spark.range(1).write.format("com.databricks.spark.csv.CsvRelation").save("/tmp/abc") java.lang.InstantiationException: com.databricks.spark.csv.CsvRelation ... ``` ```scala scala> spark.range(1).write.format("com.databricks.spark.csv.CsvRelatio").save("/tmp/abc") java.lang.ClassNotFoundException: Failed to find data source: com.databricks.spark.csv.CsvRelatio. Please find packages at http://spark.apache.org/third-party-projects.html ... ``` Author: hyukjinkwon <gurwls223@gmail.com> Closes apache#17916 from HyukjinKwon/datasource-detect.

xy1024xiangyu · 2021-03-17T09:36:41Z

@HyukjinKwon @cloud-fan , according to the discussion, it seemed that the "Multiple sources found for csv" issue has been solved. However, when I running my Java jar, an error happens.
The Java code is as follows:
DataFrameReader read = spark.read()；
JavaRDD<String> stringJavaRDD = read.textFile(inputPath).javaRDD();

When running the Java code in IDE, the program works well. However when using spark-submit, the error as follows:

org.apache.spark.sql.AnalysisException: Multiple sources found for text (org.apache.spark.sql.execution.datasources.v2.text.TextDataSourceV2, org.apache.spark.sql.execution.datasources.text.TextFileFormat), please specify the fully qualified class name.; at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:707) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:733) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:248) at org.apache.spark.sql.DataFrameReader.text(DataFrameReader.scala:843) at org.apache.spark.sql.DataFrameReader.textFile(DataFrameReader.scala:880) at org.apache.spark.sql.DataFrameReader.textFile(DataFrameReader.scala:852) at com.three2three.bigfoot.vola.NormalizeSnapshotSigmaAxisImpliedVola.main(NormalizeSnapshotSigmaAxisImpliedVola.java:306) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Even, I change my code to
DataFrameReader read = spark.read()；
JavaRDD<String> stringJavaRDD = read.format("org.apache.spark.sql.execution.datasources.text.TextFileFormat").textFile(inputPath).javaRDD(); does not help with this problem.

Detailed description here: https://stackoverflow.com/questions/66664181/spark-multiple-sources-found-for-text

Any idea how to solve this problem?

Does upgrading the installed spark version to the latest version help solve the problem?

cloud-fan · 2021-03-17T13:23:51Z

Did you closely follow the doc to run spark-submit? https://spark.apache.org/docs/latest/submitting-applications.html Especially this part When creating assembly jars, list Spark and Hadoop as provided dependencies; these need not be bundled since they are provided by the cluster manager at runtime.

xy1024xiangyu · 2021-03-17T20:43:42Z

@cloud-fan , yes, I have followed the instruction. When I running spark-submit on my standalone windows machine, such error happens. However if I put my java jar on linux server, with the same way of running spark-submit , it works well.

Why it happens? Because some path is in the system path of windows and spark-submit find two datasource? Is it a bug then. I saw many post about this "Mutilple source found for ...", e.g. csv/json. My case is text. No idea why this error happens

cloud-fan · 2021-03-18T05:01:03Z

If it only fails with Windows, it's probably a bug, but I have no idea what happens...

Different approach for SPARK-20590

03dc0f6

Fix the tests too

2dce84c

cloud-fan reviewed May 9, 2017

View reviewed changes

HyukjinKwon added 2 commits May 9, 2017 22:00

Add tests for the detection for external datasources, external/intern…

741c913

…al datasources and internal datasources

Pretty name

8c40eab

Address the comment

4450da7

HyukjinKwon commented May 9, 2017

View reviewed changes

sameeragarwal reviewed May 9, 2017

View reviewed changes

sameeragarwal mentioned this pull request May 9, 2017

[SPARK-20590] Map default input data source formats to inlined classes #17847

Closed

cloud-fan reviewed May 10, 2017

View reviewed changes

HyukjinKwon added 2 commits May 10, 2017 09:54

Improve warning message

7a464ad

Thrwos an analysis exception instead

96cf1a9

asfgit closed this in 3d2131a May 10, 2017

HyukjinKwon deleted the datasource-detect branch January 2, 2018 03:42

HyukjinKwon mentioned this pull request Jul 26, 2018

[SPARK-24924][SQL] Add mapping for built-in Avro data source #21878

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-20590][SQL] Use Spark internal datasource if multiples are found for the same shorten name #17916

[SPARK-20590][SQL] Use Spark internal datasource if multiples are found for the same shorten name #17916

HyukjinKwon commented May 9, 2017 •

edited

Loading

HyukjinKwon commented May 9, 2017

SparkQA commented May 9, 2017

cloud-fan May 9, 2017

SparkQA commented May 9, 2017

SparkQA commented May 9, 2017

SparkQA commented May 9, 2017

HyukjinKwon commented May 9, 2017

SparkQA commented May 9, 2017

sameeragarwal commented May 9, 2017

HyukjinKwon commented May 9, 2017

HyukjinKwon May 9, 2017

sameeragarwal May 9, 2017

sameeragarwal commented May 9, 2017

HyukjinKwon commented May 9, 2017

cloud-fan May 10, 2017

HyukjinKwon May 10, 2017

cloud-fan commented May 10, 2017

SparkQA commented May 10, 2017

cloud-fan commented May 10, 2017

viirya commented May 10, 2017

HyukjinKwon commented May 10, 2017

viirya commented May 10, 2017

SparkQA commented May 10, 2017

SparkQA commented May 10, 2017

cloud-fan commented May 10, 2017

HyukjinKwon commented May 10, 2017

chrishfish commented May 16, 2017

xy1024xiangyu commented Mar 17, 2021 •

edited

Loading

cloud-fan commented Mar 17, 2021

xy1024xiangyu commented Mar 17, 2021

cloud-fan commented Mar 18, 2021

[SPARK-20590][SQL] Use Spark internal datasource if multiples are found for the same shorten name #17916

[SPARK-20590][SQL] Use Spark internal datasource if multiples are found for the same shorten name #17916

Conversation

HyukjinKwon commented May 9, 2017 • edited Loading

What changes were proposed in this pull request?

How was this patch tested?

HyukjinKwon commented May 9, 2017

SparkQA commented May 9, 2017

cloud-fan May 9, 2017

Choose a reason for hiding this comment

SparkQA commented May 9, 2017

SparkQA commented May 9, 2017

SparkQA commented May 9, 2017

HyukjinKwon commented May 9, 2017

SparkQA commented May 9, 2017

sameeragarwal commented May 9, 2017

HyukjinKwon commented May 9, 2017

HyukjinKwon May 9, 2017

Choose a reason for hiding this comment

sameeragarwal May 9, 2017

Choose a reason for hiding this comment

sameeragarwal commented May 9, 2017

HyukjinKwon commented May 9, 2017

cloud-fan May 10, 2017

Choose a reason for hiding this comment

HyukjinKwon May 10, 2017

Choose a reason for hiding this comment

cloud-fan commented May 10, 2017

SparkQA commented May 10, 2017

cloud-fan commented May 10, 2017

viirya commented May 10, 2017

HyukjinKwon commented May 10, 2017

viirya commented May 10, 2017

SparkQA commented May 10, 2017

SparkQA commented May 10, 2017

cloud-fan commented May 10, 2017

HyukjinKwon commented May 10, 2017

chrishfish commented May 16, 2017

xy1024xiangyu commented Mar 17, 2021 • edited Loading

cloud-fan commented Mar 17, 2021

xy1024xiangyu commented Mar 17, 2021

cloud-fan commented Mar 18, 2021

HyukjinKwon commented May 9, 2017 •

edited

Loading

xy1024xiangyu commented Mar 17, 2021 •

edited

Loading