Skip to content

Commit

Permalink
[Hot Fix scala#42] Do not stop SparkUI if bind() is not called
Browse files Browse the repository at this point in the history
This is a bug fix for scala#42 (79d07d6).

In Master, we do not bind() each SparkUI because we do not want to start a server for each finished application. However, when we remove the associated application, we call stop() on the SparkUI, which throws an assertion failure.

This fix ensures we don't call stop() on a SparkUI that was never bind()'ed.

Author: Andrew Or <andrewor14@gmail.com>

Closes scala#188 from andrewor14/ui-fix and squashes the following commits:

94a925f [Andrew Or] Do not stop SparkUI if bind() is not called
  • Loading branch information
andrewor14 authored and pwendell committed Mar 20, 2014
1 parent 66a03e5 commit ca76423
Showing 1 changed file with 2 additions and 9 deletions.
11 changes: 2 additions & 9 deletions core/src/main/scala/org/apache/spark/deploy/master/Master.scala
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,6 @@ private[spark] class Master(

override def postStop() {
webUi.stop()
appIdToUI.values.foreach(_.stop())
masterMetricsSystem.stop()
applicationMetricsSystem.stop()
persistenceEngine.close()
Expand Down Expand Up @@ -622,10 +621,7 @@ private[spark] class Master(
if (completedApps.size >= RETAINED_APPLICATIONS) {
val toRemove = math.max(RETAINED_APPLICATIONS / 10, 1)
completedApps.take(toRemove).foreach( a => {
appIdToUI.remove(a.id).foreach { ui =>
ui.stop()
webUi.detachUI(ui)
}
appIdToUI.remove(a.id).foreach { ui => webUi.detachUI(ui) }
applicationMetricsSystem.removeSource(a.appSource)
})
completedApps.trimStart(toRemove)
Expand Down Expand Up @@ -681,10 +677,7 @@ private[spark] class Master(
// Do not call ui.bind() to avoid creating a new server for each application
ui.start()
val success = replayerBus.replay(eventLogDir)
if (!success) {
ui.stop()
None
} else Some(ui)
if (success) Some(ui) else None
}

/** Generate a new app ID given a app's submission date */
Expand Down

0 comments on commit ca76423

Please sign in to comment.