[SPARK-16304] LinkageError should not crash Spark executor #13982
Conversation
|
Test build #3149 has finished for PR 13982 at commit
|
|
cc @JoshRosen and @ericl |
| @@ -1872,6 +1872,12 @@ private[spark] object Utils extends Logging { | |||
| } | |||
| } | |||
|
|
|||
| /** Returns true if the given exception is a linkage error. */ | |||
| def isLinkageError(e: Throwable): Boolean = e match { | |||
| case _: java.lang.LinkageError => true | |||
There was a problem hiding this comment.
This can just be e.instanceOf[LinkageError] but an entire method for this is not appropriate
|
Test build #3160 has finished for PR 13982 at commit
|
|
@srowen any more feedback? |
|
I don't feel strongly about it. It's something easy to revisit later to change if it turns out special-casing this is undesirable. |
|
Alright merging in master. Thanks. |
|
@rxin any objection to merge this one into 2.0? I noticed sometimes spark-shell will throw |
|
No objection. On Wednesday, October 26, 2016, Shixiong Zhu notifications@github.com
|
|
thanks! Merging to 2.0. |
What changes were proposed in this pull request?
This patch updates the failure handling logic so Spark executor does not crash when seeing LinkageError.
How was this patch tested?
Added an end-to-end test in FailureSuite.