Skip to content

Commit

Permalink
Fix flaky verifyPeakExecutionMemorySet
Browse files Browse the repository at this point in the history
Previously we would assert things before all events are guaranteed
to have been processed. To fix this, just block until all events
are actually processed, i.e. until the listener queue is empty.
  • Loading branch information
Andrew Or committed Jan 29, 2016
1 parent c5f745e commit d67a529
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions core/src/test/scala/org/apache/spark/AccumulatorSuite.scala
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,8 @@ private[spark] object AccumulatorSuite {
val listener = new SaveInfoListener
sc.addSparkListener(listener)
testBody
// wait until all events have been processed before proceeding to assert things
sc.listenerBus.waitUntilEmpty(10 * 1000)
val accums = listener.getCompletedStageInfos.flatMap(_.accumulables.values)
val isSet = accums.exists { a =>
a.name == Some(PEAK_EXECUTION_MEMORY) && a.value.exists(_.asInstanceOf[Long] > 0L)
Expand Down

0 comments on commit d67a529

Please sign in to comment.