-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-23830][YARN] added check to ensure main method is found #21168
Conversation
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
Outdated
Show resolved
Hide resolved
The exception happens because |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please fix the title to follow the convention:
[BUG-12345][yarn] Summary.
finish(FinalApplicationStatus.SUCCEEDED, ApplicationMaster.EXIT_SUCCESS) | ||
logDebug("Done running users class") | ||
if(!Modifier.isStatic(mainMethod.getModifiers)) { | ||
logError(s"Could not find main method in object ${args.userClass}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The message could be a little more helpful here.
mainMethod.invoke(null, userArgs.toArray) | ||
finish(FinalApplicationStatus.SUCCEEDED, ApplicationMaster.EXIT_SUCCESS) | ||
logDebug("Done running users class") | ||
if(!Modifier.isStatic(mainMethod.getModifiers)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This won't pass the style checker...
ok to test |
Test build #89910 has finished for PR 21168 at commit
|
Test build #89911 has finished for PR 21168 at commit
|
The change is fail to build, please fix it. |
Also this is a yarn, not core, change. |
added an import for `java.lang.reflect.Modifier`
Test build #89913 has finished for PR 21168 at commit
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Merging to master branch.
What changes were proposed in this pull request?
When a user specifies the wrong class -- or, in fact, a class instead of an object -- Spark throws an NPE which is not useful for debugging. This was reported in SPARK-23830. This PR adds a check to ensure the main method was found and logs a useful error in the event that it's null.
How was this patch tested?