Skip to content

Commit

Permalink
[SPARK-25266][CORE] Fix memory leak in Barrier Execution Mode
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

BarrierCoordinator uses Timer and TimerTask. `TimerTask#cancel()` is invoked in ContextBarrierState#cancelTimerTask but `Timer#purge()` is never invoked.

Once a TimerTask is scheduled, the reference to it is not released until `Timer#purge()` is invoked even though `TimerTask#cancel()` is invoked.

## How was this patch tested?

I checked the number of instances related to the TimerTask using jmap.

Closes apache#22258 from sarutak/fix-barrierexec-oom.

Authored-by: sarutak <sarutak@oss.nttdata.co.jp>
Signed-off-by: Xiangrui Meng <meng@databricks.com>
  • Loading branch information
sarutak authored and fjh100456 committed Aug 31, 2018
1 parent 0c39ead commit 0544276
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ private[spark] class BarrierCoordinator(
private def cancelTimerTask(): Unit = {
if (timerTask != null) {
timerTask.cancel()
timer.purge()
timerTask = null
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ class BarrierTaskContext(
throw e
} finally {
timerTask.cancel()
timer.purge()
}
}

Expand Down

0 comments on commit 0544276

Please sign in to comment.