Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-20728][SQL][FOLLOWUP] Use an actionable exception message #19903

Closed
wants to merge 4 commits into from
Closed

[SPARK-20728][SQL][FOLLOWUP] Use an actionable exception message #19903

wants to merge 4 commits into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Dec 6, 2017

What changes were proposed in this pull request?

This is a follow-up of #19871 to improve an exception message.

How was this patch tested?

Pass the Jenkins.

@cloud-fan
Copy link
Contributor

LGTM

@@ -602,7 +602,7 @@ object DataSource extends Logging {
provider1.startsWith("org.apache.spark.sql.hive.orc")) {
throw new AnalysisException(
"Hive-based ORC data source must be used with Hive support enabled. " +
"Please use native ORC data source instead")
"Please use native ORC data source instead by `SET spark.sql.orc.impl=native`")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait .. @dongjoon-hyun can we just say like by setting 'spark.sql.orc.impl' configuration to 'native'?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-20728][SQL][FOLLOWUP] Use a actionable exception message [SPARK-20728][SQL][FOLLOWUP] Use an actionable exception message Dec 6, 2017
@HyukjinKwon
Copy link
Member

LGTM

@dongjoon-hyun
Copy link
Member Author

Thank you, @HyukjinKwon and @cloud-fan .

@@ -602,7 +602,8 @@ object DataSource extends Logging {
provider1.startsWith("org.apache.spark.sql.hive.orc")) {
throw new AnalysisException(
"Hive-based ORC data source must be used with Hive support enabled. " +
"Please use native ORC data source instead")
"Please use native ORC data source instead by setting 'spark.sql.orc.impl' " +
"configuration to 'native'")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the native ORC data source by setting 'spark.sql.orc.impl' to 'native'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep.

@@ -602,7 +602,8 @@ object DataSource extends Logging {
provider1.startsWith("org.apache.spark.sql.hive.orc")) {
throw new AnalysisException(
"Hive-based ORC data source must be used with Hive support enabled. " +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hive built-in ORC data source must be used with Hive support enabled.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep.

@gatorsmile
Copy link
Member

Do you have a test case?

@dongjoon-hyun
Copy link
Member Author

Yep. The test case will be updated together.

@dongjoon-hyun
Copy link
Member Author

@cloud-fan , @HyukjinKwon , @gatorsmile . The PR is updated again.
Please let me know if I need to update more. Thank you!

@SparkQA
Copy link

SparkQA commented Dec 6, 2017

Test build #84521 has finished for PR 19903 at commit d6cdde7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile
Copy link
Member

LGTM pending Jenkins on $84528

@dongjoon-hyun
Copy link
Member Author

Thanks, @gatorsmile .

@SparkQA
Copy link

SparkQA commented Dec 6, 2017

Test build #84526 has finished for PR 19903 at commit ba1e3ae.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 6, 2017

Test build #84528 has finished for PR 19903 at commit d7d268d.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member Author

Retest this please.

@SparkQA
Copy link

SparkQA commented Dec 6, 2017

Test build #84532 has finished for PR 19903 at commit d7d268d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

retest this please

@@ -601,8 +601,9 @@ object DataSource extends Logging {
if (provider1.toLowerCase(Locale.ROOT) == "orc" ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As suggested in other PR, we can remove this condition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. I'll remove this here since it's small.

@viirya
Copy link
Member

viirya commented Dec 6, 2017

LGTM with one minor comment.

@SparkQA
Copy link

SparkQA commented Dec 6, 2017

Test build #84536 has finished for PR 19903 at commit d7d268d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 6, 2017

Test build #84541 has finished for PR 19903 at commit 9809af8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

Merged to master.

@asfgit asfgit closed this in fb6a922 Dec 6, 2017
@dongjoon-hyun
Copy link
Member Author

Thank you, @HyukjinKwon , @cloud-fan , @gatorsmile , @viirya !

@dongjoon-hyun dongjoon-hyun deleted the orc_exception branch December 6, 2017 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants