Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-40180][SQL] Format error messages by spark-sql #37590

Closed
wants to merge 6 commits into from

Conversation

MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Aug 20, 2022

What changes were proposed in this pull request?

  1. Respect the SQL config spark.sql.error.messageFormat introduced by [SPARK-40098][SQL] Format error messages in the Thrift Server #37520, and output error messages in the one of format: PRETTY (by default), MINIMAL or STANDARD.
  2. In the PRETTY format, output the error message of the AnalysisException exception in the same way as for other exceptions, i. e. w/o Error in query:.
  3. Take into account the silent CLI option, and output the call stack only when it is false.

In the MINIMAL and STANDARD formats don't print the call stack independently from the silent mode.

Why are the changes needed?

To respect the SQL config spark.sql.error.messageFormat and to be consistent to error outputs of the Thrift Server.

Does this PR introduce any user-facing change?

Yes.

The PR changes the behavior for AnalysisException. In that case, spark-sql does not output the prefix: Error in query: by default (format is PRETTY).

Before:

spark-sql> DROP TABLE ajhkdha;
Error in query: Table or view not found: ajhkdha; line 1 pos 11;

After:

spark-sql> DROP TABLE ajhkdha;
Table or view not found: ajhkdha; line 1 pos 11;

How was this patch tested?

By running the modified test suites:

$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly org.apache.spark.sql.hive.thriftserver.CliSuite"

@github-actions github-actions bot added the SQL label Aug 20, 2022
@MaxGekk MaxGekk changed the title [WIP][SQL] Format error messages by spark-sql [WIP][SPARK-40180][SQL] Format error messages by spark-sql Aug 22, 2022
@MaxGekk MaxGekk changed the title [WIP][SPARK-40180][SQL] Format error messages by spark-sql [SPARK-40180][SQL] Format error messages by spark-sql Aug 22, 2022
@MaxGekk MaxGekk marked this pull request as ready for review August 22, 2022 20:34
@github-actions github-actions bot added the DOCS label Aug 22, 2022
@MaxGekk
Copy link
Member Author

MaxGekk commented Aug 22, 2022

@srielau @anchovYu Please, take a look at the PR.

Copy link
Contributor

@srielau srielau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

// to the console.
case _ => err.println(s"""Error in query: ${e.getMessage}""")
}
case _ => err.println(rc.getErrorMessage())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even before this PR, we may not print Error in query: either. I think it's fine to clean it up and get rid of this prefix

@MaxGekk
Copy link
Member Author

MaxGekk commented Aug 24, 2022

Merging to master. Thank you, @srielau @yaooqinn @cloud-fan for review.

@MaxGekk MaxGekk closed this in 7046978 Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants