ContainerProxyTests Heisenbug

[chetanmeh]

Seen failure in ContainerProxyTest at https://scans.gradle.com/s/4lb3ctstu5cqc/tests/xaqugzlgs2zxa-igyfqphtwknzs?openStackTraces=WzBd in ContainerProxy should run an action after pausing the container

java.util.NoSuchElementException: None.getClose stacktrace
at scala.None$.get(Option.scala:349)
at scala.None$.get(Option.scala:347)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.$anonfun$new$11(ContainerProxyTests.scala:379)
at akka.testkit.TestKitBase.poll$2(TestKit.scala:312)
at akka.testkit.TestKitBase.awaitAssert(TestKit.scala:329)
at akka.testkit.TestKitBase.awaitAssert$(TestKit.scala:301)
at akka.testkit.TestKit.awaitAssert(TestKit.scala:896)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.$anonfun$new$10(ContainerProxyTests.scala:366)
at akka.testkit.TestKitBase.within(TestKit.scala:360)
at akka.testkit.TestKitBase.within$(TestKit.scala:348)
at akka.testkit.TestKit.within(TestKit.scala:896)
at akka.testkit.TestKitBase.within(TestKit.scala:374)
at akka.testkit.TestKitBase.within$(TestKit.scala:374)
at akka.testkit.TestKit.within(TestKit.scala:896)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.$anonfun$new$9(ContainerProxyTests.scala:340)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1682)
at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.withFixture(ContainerProxyTests.scala:49)
at org.scalatest.FlatSpecLike.invokeWithFixture$1(FlatSpecLike.scala:1680)
at org.scalatest.FlatSpecLike.$anonfun$runTest$1(FlatSpecLike.scala:1692)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)

[2018-11-28T12:01:35.803Z] [INFO] [#tid_sid_testing] [ContainerProxy]  [marker:invoker_collectLogs_start:194]
[2018-11-28T12:01:35.804Z] [INFO] [#tid_sid_testing] [ContainerProxy]  [marker:invoker_collectLogs_finish:194:0]
[2018-11-28T12:01:35.808Z] [INFO] [#tid_sid_testing] [ContainerProxy]  [marker:invoker_collectLogs_start:199]
[2018-11-28T12:01:35.809Z] [INFO] [#tid_sid_testing] [ContainerProxy]  [marker:invoker_collectLogs_finish:199:0]

[chetanmeh]

Another variant https://scans.gradle.com/s/vpweyhb5js672/tests/xaqugzlgs2zxa-hjzpsz5dks2rq in ContainerProxy should complete the transaction and reuse the container on a failed run IFF failure was applicationError

ContainerProxy should complete the transaction and reuse the container on a failed run IFF failure was applicationError (5.067s)
org.scalatest.exceptions.TestFailedException: Some(200) was not equal to Some(250)Close stacktrace
at org.scalatest.MatchersHelper$.indicateFailure(MatchersHelper.scala:340)
at org.scalatest.Matchers$AnyShouldWrapper.shouldBe(Matchers.scala:6864)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.$anonfun$new$25(ContainerProxyTests.scala:631)
at akka.testkit.TestKitBase.poll$2(TestKit.scala:312)
at akka.testkit.TestKitBase.awaitAssert(TestKit.scala:329)
at akka.testkit.TestKitBase.awaitAssert$(TestKit.scala:301)
at akka.testkit.TestKit.awaitAssert(TestKit.scala:896)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.$anonfun$new$24(ContainerProxyTests.scala:620)
at akka.testkit.TestKitBase.within(TestKit.scala:360)
at akka.testkit.TestKitBase.within$(TestKit.scala:348)
at akka.testkit.TestKit.within(TestKit.scala:896)
at akka.testkit.TestKitBase.within(TestKit.scala:374)
at akka.testkit.TestKitBase.within$(TestKit.scala:374)
at akka.testkit.TestKit.within(TestKit.scala:896)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.$anonfun$new$23(ContainerProxyTests.scala:579)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1682)
at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.withFixture(ContainerProxyTests.scala:49)
at org.scalatest.FlatSpecLike.invokeWithFixture$1(FlatSpecLike.scala:1680)
at org.scalatest.FlatSpecLike.$anonfun$runTest$1(FlatSpecLike.scala:1692)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
at org.scalatest.FlatSpecLike.runTest(FlatSpecLike.scala:1692)
at org.scalatest.FlatSpecLike.runTest$(FlatSpecLike.scala:1674)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.runTest(ContainerProxyTests.scala:49)
at org.scalatest.FlatSpecLike.$anonfun$runTests$1(FlatSpecLike.scala:1750)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:396)
at scala.collection.immutable.List.foreach(List.scala:388)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:373)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:410)
at scala.collection.immutable.List.foreach(List.scala:388)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:379)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
at org.scalatest.FlatSpecLike.runTests(FlatSpecLike.scala:1750)
at org.scalatest.FlatSpecLike.runTests$(FlatSpecLike.scala:1749)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.runTests(ContainerProxyTests.scala:49)
at org.scalatest.Suite.run(Suite.scala:1147)
at org.scalatest.Suite.run$(Suite.scala:1129)
at org.apache.openwhisk.core.containerpool.test.ContainerProxyTests.org$scalatest$FlatSpecLike$$super$run(ContainerProxyTests.scala:49)

[chetanmeh]

Failure at https://scans.gradle.com/s/tzrqacw53zadg/tests/xaqugzlgs2zxa-igyfqphtwknzs ContainerProxy should run an action after pausing the container

[chetanmeh]

Lot more failures at https://scans.gradle.com/s/nim4ejomdzfy2/tests/failed

[chetanmeh]

Another one at https://scans.gradle.com/s/st23hp56clclk/tests/xaqugzlgs2zxa-4fdg7fshdyxgs

[chetanmeh]

Some improvements made in tests via #4289