Skip to content

Commit

Permalink
[SPARK-44542][CORE] Eagerly load SparkExitCode class in exception han…
Browse files Browse the repository at this point in the history
…dler

### What changes were proposed in this pull request?
1. eagerly load SparkExitCode class in the the SparkUncaughtExceptionHandler

### Why are the changes needed?
In some extreme case, it's possible for SparkUncaughtExceptionHandler's exit/halt process function calls throw
an exception if the SparkExitCode is not loaded earlier, See corresponding jira: [SPARK-44542](https://issues.apache.org/jira/browse/SPARK-44542) for more details.

By eagerly load SparkExitCode class, we can make sure at least the halt/exit would work properly.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
No logic change, hence no new UTs.

Closes #42195 from advancedxy/SPARK-44542.

Authored-by: Xianjin <xianjin@apache.org>
Signed-off-by: Sean Owen <srowen@gmail.com>
(cherry picked from commit 32498b3)
Signed-off-by: Sean Owen <srowen@gmail.com>
  • Loading branch information
advancedxy authored and srowen committed Jul 31, 2023
1 parent 455ffd3 commit 47224b3
Showing 1 changed file with 6 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ import org.apache.spark.internal.Logging
private[spark] class SparkUncaughtExceptionHandler(val exitOnUncaughtException: Boolean = true)
extends Thread.UncaughtExceptionHandler with Logging {

locally {
// eagerly load SparkExitCode class, so the System.exit and runtime.halt have a chance to be
// executed when the disk containing Spark jars is corrupted. See SPARK-44542 for more details.
val _ = SparkExitCode.OOM
}

override def uncaughtException(thread: Thread, exception: Throwable): Unit = {
try {
// Make it explicit that uncaught exceptions are thrown when container is shutting down.
Expand Down

0 comments on commit 47224b3

Please sign in to comment.