Skip to content

Commit

Permalink
[SPARK-13693][STREAMING][TESTS] Stop StreamingContext before deleting…
Browse files Browse the repository at this point in the history
… checkpoint dir

## What changes were proposed in this pull request?

Stop StreamingContext before deleting checkpoint dir to avoid the race condition that deleting the checkpoint dir and writing checkpoint happen at the same time.

The flaky test log is here: https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7/256/testReport/junit/org.apache.spark.streaming/MapWithStateSuite/_It_is_not_a_test_/

## How was this patch tested?

unit tests

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #11531 from zsxwing/SPARK-13693.
  • Loading branch information
zsxwing committed Mar 5, 2016
1 parent adce5ee commit 8290004
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,10 @@ class MapWithStateSuite extends SparkFunSuite
}

after {
StreamingContext.getActive().foreach { _.stop(stopSparkContext = false) }
if (checkpointDir != null) {
Utils.deleteRecursively(checkpointDir)
}
StreamingContext.getActive().foreach { _.stop(stopSparkContext = false) }
}

override def beforeAll(): Unit = {
Expand Down

0 comments on commit 8290004

Please sign in to comment.