-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-3658][SQL]Take thrift server as a daemon #2509
Conversation
QA tests have started for PR 2509 at commit
|
QA tests have finished for PR 2509 at commit
|
Test PASSed. |
@liancheng - can you take a look at this one? |
if (childMainClass.contains("thriftserver")) { | ||
println(s"Failed to load main class $childMainClass.") | ||
println("You need to build Spark with -Phive.") | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel complicated to add this logic here... SparkSubmit
should not be coupled with Thrift server, but it seems that this is the only legitimate place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After some thoughts, I think it's OK to add this checking here. Mainly because it avoids using the exit code checking trick, which may cause potential exit code conflict.
Generally this is a good idea. But it would be better to make |
QA tests have started for PR 2509 at commit
|
@liancheng As you said, I put "spark-submit" as an option to achieve generalization and use source(dot) instead of As for the ClassNotFound error message, we could either do like this or just delete it? please check.thanks |
QA tests have finished for PR 2509 at commit
|
Test PASSed. |
@@ -0,0 +1,25 @@ | |||
#!/usr/bin/env bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file should be executable, please chmod +x
.
Thanks for working on this! I tested this PR locally and it works fine. But there are still some minor issues pending to be resolved, please refer to the comments for details. |
@liancheng I tried |
Test FAILed. |
QA tests have started for PR 2509 at commit
|
After updating my local repo, I found that |
Tests timed out after a configured wait of |
eh...I cloned the repository on another laptop and found it's executable, as shown in top-left corner of https://github.com/WangTaoTheTonic/spark/blob/thriftserver/sbin/stop-thriftserver.sh. Could you verify this again? |
Ah, sorry, my fault. Then this LGTM, thanks! |
Jenkins, test this please. |
QA tests have started for PR 2509 at commit
|
QA tests have finished for PR 2509 at commit
|
Test PASSed. |
Thanks! I've merged this to master. |
https://issues.apache.org/jira/browse/SPARK-3658
And keep the
CLASS_NOT_FOUND_EXIT_STATUS
and exit message inSparkSubmit.scala
.