Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-35062][SQL] Group exception messages in sql/streaming #32464

Closed
wants to merge 7 commits into from
Closed

[SPARK-35062][SQL] Group exception messages in sql/streaming #32464

wants to merge 7 commits into from

Conversation

beliefer
Copy link
Contributor

@beliefer beliefer commented May 7, 2021

What changes were proposed in this pull request?

This PR group exception messages in sql/core/src/main/scala/org/apache/spark/sql/streaming.

Why are the changes needed?

It will largely help with standardization of error messages and its maintenance.

Does this PR introduce any user-facing change?

No. Error messages remain unchanged.

How was this patch tested?

No new tests - pass all original tests to make sure it doesn't break any existing behavior.

@SparkQA
Copy link

SparkQA commented May 7, 2021

Test build #138232 has finished for PR 32464 at commit 23c7f91.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 7, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42754/

@SparkQA
Copy link

SparkQA commented May 7, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42757/

@SparkQA
Copy link

SparkQA commented May 7, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42757/

@SparkQA
Copy link

SparkQA commented May 7, 2021

Test build #138235 has finished for PR 32464 at commit ce9d446.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 7, 2021

Test build #138249 has finished for PR 32464 at commit a615825.

  • This patch fails to build.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 7, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42771/

@SparkQA
Copy link

SparkQA commented May 8, 2021

Test build #138277 has finished for PR 32464 at commit c893aa5.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 8, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42799/

@SparkQA
Copy link

SparkQA commented May 8, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42800/

@SparkQA
Copy link

SparkQA commented May 8, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42800/

@SparkQA
Copy link

SparkQA commented May 8, 2021

Test build #138278 has finished for PR 32464 at commit bab4787.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@beliefer
Copy link
Contributor Author

beliefer commented May 8, 2021

ping @allisonwang-db

allisonwang-db
allisonwang-db reviewed May 10, 2021
View reviewed changes
Copy link
Contributor

@allisonwang-db allisonwang-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! There are two more exceptions under streaming/ui. How about adding them in the same PR?

'sql/core/src/main/scala/org/apache/spark/sql/streaming/ui'

Filename Count
StreamingQueryPage.scala 1
StreamingQueryStatisticsPage.scala 1

sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala Outdated
@@ -1391,4 +1391,58 @@ private[spark] object QueryCompilationErrors {
def functionUnsupportedInV2CatalogError(): Throwable = {
new AnalysisException("function is only supported in v1 catalog")
}

def operateHiveDataSourceDirectlyError(operation: String): Throwable = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about cannotOperateOnHiveDataSourceError

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about cannotOperateOnHiveDataSourceFilesError ?

@beliefer
Copy link
Contributor Author

Looks good! There are two more exceptions under streaming/ui. How about adding them in the same PR?

'sql/core/src/main/scala/org/apache/spark/sql/streaming/ui'

Filename Count
StreamingQueryPage.scala 1
StreamingQueryStatisticsPage.scala 1

I checked the two file and found all the errors is assert-like Exception. According to the discussion between @cloud-fan and me, we not need to treat the assert-like Exception.

@SparkQA
Copy link

SparkQA commented May 11, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42875/

@SparkQA
Copy link

SparkQA commented May 11, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42875/

@SparkQA
Copy link

SparkQA commented May 11, 2021

Test build #138352 has finished for PR 32464 at commit 4284458.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@beliefer
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented May 11, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42884/

@SparkQA
Copy link

SparkQA commented May 11, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42884/

@SparkQA
Copy link

SparkQA commented May 11, 2021

Test build #138361 has finished for PR 32464 at commit 4284458.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@beliefer
Copy link
Contributor Author

ping @allisonwang-db

@beliefer
Copy link
Contributor Author

cc @cloud-fan

@cloud-fan
Copy link
Contributor

thanks, merging to master!

@cloud-fan cloud-fan closed this in c2e15cc May 13, 2021
@beliefer
Copy link
Contributor Author

@allisonwang-db Thanks for you review. @cloud-fan Thank you too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@allisonwang-db allisonwang-db allisonwang-db left review comments

Assignees
No one assigned
Labels
SQL STRUCTURED STREAMING
Projects
None yet
Milestone
No milestone
4 participants
@beliefer @SparkQA @cloud-fan @allisonwang-db