Skip to content

Commit

Permalink
[SPARK-23962][SQL][TEST] Fix race in currentExecutionIds().
Browse files Browse the repository at this point in the history
SQLMetricsTestUtils.currentExecutionIds() was racing with the listener
bus, which lead to some flaky tests.  We should wait till the listener bus is
empty.

I tested by adding some Thread.sleep()s in SQLAppStatusListener, which
reproduced the exceptions I saw on Jenkins.  With this change, they went
away.

Author: Imran Rashid <irashid@cloudera.com>

Closes #21041 from squito/SPARK-23962.
  • Loading branch information
squito authored and cloud-fan committed Apr 12, 2018
1 parent e904dfa commit 6a2289e
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ trait SQLMetricsTestUtils extends SQLTestUtils {
import testImplicits._

protected def currentExecutionIds(): Set[Long] = {
spark.sparkContext.listenerBus.waitUntilEmpty(10000)
statusStore.executionsList.map(_.executionId).toSet
}

Expand Down

0 comments on commit 6a2289e

Please sign in to comment.