Skip to content

Commit

Permalink
[SPARK-6314] [CORE] handle JsonParseException for history server
Browse files Browse the repository at this point in the history
This is handled in the same way with [SPARK-6197](https://issues.apache.org/jira/browse/SPARK-6197). The result of this PR is that exception showed in history server log will be replaced by a warning, and the application that with un-complete history log file will be listed on history server webUI

Author: Zhang, Liye <liye.zhang@intel.com>

Closes #5736 from liyezhang556520/SPARK-6314 and squashes the following commits:

b8d2d88 [Zhang, Liye] handle JsonParseException for history server
  • Loading branch information
liyezhang556520 authored and Andrew Or committed Apr 28, 2015
1 parent 80ba82e commit 37fb5a9
Showing 1 changed file with 3 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -256,8 +256,9 @@ private[history] class FsHistoryProvider(conf: SparkConf) extends ApplicationHis
}
try {
val appListener = new ApplicationEventListener
val appCompleted = isApplicationCompleted(eventLog)
bus.addListener(appListener)
bus.replay(logInput, logPath.toString)
bus.replay(logInput, logPath.toString, !appCompleted)
new FsApplicationHistoryInfo(
logPath.getName(),
appListener.appId.getOrElse(logPath.getName()),
Expand All @@ -266,7 +267,7 @@ private[history] class FsHistoryProvider(conf: SparkConf) extends ApplicationHis
appListener.endTime.getOrElse(-1L),
getModificationTime(eventLog).get,
appListener.sparkUser.getOrElse(NOT_STARTED),
isApplicationCompleted(eventLog))
appCompleted)
} finally {
logInput.close()
}
Expand Down

0 comments on commit 37fb5a9

Please sign in to comment.