Skip to content
Browse files

[SPARK-19688][STREAMING] Not to read `spark.yarn.credentials.file` fr…

…om checkpoint.

## What changes were proposed in this pull request?

Reload the `spark.yarn.credentials.file` property when restarting a streaming application from checkpoint.

## How was this patch tested?

Manual tested with 1.6.3 and 2.1.1.
I didn't test this with master because of some compile problems, but I think it will be the same result.

## Notice

This should be merged into maintenance branches too.

jira: [SPARK-21008](

Author: saturday_s <>

Closes #18230 from saturday-shi/SPARK-21008.

(cherry picked from commit e92ffe6)
Signed-off-by: Marcelo Vanzin <>
  • Loading branch information...
saturday_s authored and vanzin committed Jun 19, 2017
1 parent 6b315f3 commit a233fac0b8bf8229d938a24f2ede2d9d8861c284
Showing with 3 additions and 0 deletions.
  1. +3 −0 streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala
@@ -56,6 +56,9 @@ class Checkpoint(ssc: StreamingContext, val checkpointTime: Time)

val newSparkConf = new SparkConf(loadDefaults = false).setAll(sparkConfPairs)

0 comments on commit a233fac

Please sign in to comment.
You can’t perform that action at this time.