Fork in test should handle exceptions during creation of test class #653

ijuma · 2013-01-25T13:54:48Z

We recently ran into a pretty severe issue where a ScalaTest test class was throwing a NPE during construction, which caused test execution to stop but for success to be reported. This meant that we were running 1/3 of our test suite instead of the full test suite and failures were not reported. Note that this was a multi-module build.

I can provide a simple test case if that helps.

The output was:

Exception in thread "Thread-32" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2571)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
    at sbt.React.react(ForkTests.scala:98)
    at sbt.ForkTests$$anonfun$apply$2$Acceptor$2$.run(ForkTests.scala:66)
    at java.lang.Thread.run(Thread.java:722)
Exception in thread "Thread-32" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2571)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
    at sbt.React.react(ForkTests.scala:98)
    at sbt.ForkTests$$anonfun$apply$2$Acceptor$2$.run(ForkTests.scala:66)
    at java.lang.Thread.run(Thread.java:722)
[info] Passed: : Total 0, Failed 0, Errors 0, Passed 0, Skipped 0

The text was updated successfully, but these errors were encountered:

ijuma · 2013-01-25T13:55:05Z

/cc @aloiscochard

harrah · 2013-01-25T14:50:22Z

Yes, a test case usually helps fix it sooner. You are on 0.12.2, right?

ijuma · 2013-01-26T11:46:50Z

0.12.2 indeed. Will try to create an isolated test case.

ijuma · 2013-01-27T17:16:09Z

I updated a previous project with a SBT test case to throw an exception during test construction:

ijuma/sbt-test-bug@ac82782

The output using SBT 0.12.2 in my machine:

ijuma% sbt test
[info] Loading project definition from /Users/ijuma/src/sbt-test-bug/project
[info] Set current project to root (in build file:/Users/ijuma/src/sbt-test-bug/)
[info] No tests to run for root/test:test
Exception in thread "Thread-1" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2572)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
    at sbt.React.react(ForkTests.scala:98)
    at sbt.ForkTests$$anonfun$apply$2$Acceptor$2$.run(ForkTests.scala:66)
    at java.lang.Thread.run(Thread.java:722)
[info] Passed: : Total 0, Failed 0, Errors 0, Passed 0, Skipped 0
[info] No tests to run for data/test:test
[success] Total time: 1 s, completed 27-Jan-2013 17:14:31

ijuma · 2013-01-29T00:28:48Z

Thanks!

…#653.

taylorleese · 2013-10-29T03:21:37Z

I seem to be getting this error with sbt 0.13.0.

Exception in thread "Thread-3" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2577)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
    at sbt.React.react(ForkTests.scala:110)
    at sbt.ForkTests$$anonfun$mainTestTask$1$Acceptor$2$.run(ForkTests.scala:69)
    at java.lang.Thread.run(Thread.java:695)

harrah · 2013-11-04T14:51:17Z

@taylorleese certainly it is a bug, but it is unlikely to have the same cause and warrants a separate issue. Unfortunately, I'd need a test case. sbt 0.13.1 might help pinpoint the problem with more logging, but I don't know if it will do anything at this point in execution. (0.13.1-RC1 had some issues so it wasn't announced- RC2 will be out this week.)

benburton · 2013-11-12T18:36:34Z

I'm seeing this as well on sbt 0.13.0

Exception in thread "Thread-18" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2598)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1318)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    at sbt.React.react(ForkTests.scala:98)
    at sbt.ForkTests$$anonfun$apply$2$Acceptor$2$.run(ForkTests.scala:66)
    at java.lang.Thread.run(Thread.java:744)

cchantep · 2013-11-20T17:21:22Z

Same on sbt 0.13.1-RC2:

Exception in thread "Thread-54" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2577)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
    at sbt.React.react(ForkTests.scala:116)
    at sbt.ForkTests$$anonfun$mainTestTask$1$Acceptor$2$.run(ForkTests.scala:75)
    at java.lang.Thread.run(Thread.java:722)

harrah · 2013-11-20T17:25:44Z

@cchantep right, there hasn't been a new fix since the original fix. I need a test case to be able to do anything- the stack trace doesn't help much, unfortunately. The earlier comment about 0.13.1 was that it might have additional information useful for producing a test case.

cchantep · 2013-11-20T17:27:17Z

Will see if I can extract a clear test case from the erroneous involved specs test.

harrah · 2013-11-20T17:28:40Z

Thanks a lot. I'd appreciate it.

cchantep · 2013-11-20T17:42:56Z

Unfortunatly it seems several test combinations given the blocking situation, I can restrict it more than https://github.com/playframework/playframework/blob/master/framework/src/anorm/src/test/scala/anorm/AnormSpec.scala .

If you comment out at least one test, sbt execution is ok, running all is blocked (sometime, but majority of time for me today).

playframework/framework# sbt
sbt> project Anorm
sbt> test

harrah · 2013-11-20T17:44:20Z

but it is reproducible on a public codebase? that's a good enough start for me to reopen, thanks.

cchantep · 2013-11-20T17:45:24Z

yep, no need of specific fixture, checking out play master allow to reproduce it.

harrah · 2013-11-20T18:08:35Z

I don't see an EOFException, just a hang, which appears to be in the application/test code and not in sbt. The relevant stack traces from the remote jvm:

"specs2.DefaultExecutionStrategy1" prio=10 tid=0x00007fddf83e4000 nid=0xfbf in Object.wait() [0x00007fddf4de8000]
   java.lang.Thread.State: RUNNABLE
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:188)
    at anorm.H2Database$class.withConnection(H2Database.scala:11)
    at anorm.AnormSpec$.withConnection(AnormSpec.scala:9)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$1.apply(AnormSpec.scala:11)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$1.apply(AnormSpec.scala:11)
    at org.specs2.specification.ContextsInjection$$anon$1$$anonfun$asResult$1.apply(ContextsInjection.scala:11)

"specs2.DefaultExecutionStrategy2" prio=10 tid=0x00007fddf8363000 nid=0xfc0 waiting for monitor entry [0x00007fddf4ce7000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at java.sql.DriverManager.registerDriver(DriverManager.java:295)
    - waiting to lock <0x000000077bd95720> (a java.lang.Class for java.sql.DriverManager)
    at org.h2.Driver.load(Driver.java:160)
    - locked <0x000000077bdb25f0> (a java.lang.Class for org.h2.Driver)
    at org.h2.Driver.<clinit>(Driver.java:45)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:188)
    at anorm.H2Database$class.withConnection(H2Database.scala:11)
    at anorm.AnormSpec$.withConnection(AnormSpec.scala:9)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$6.apply(AnormSpec.scala:24)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$6.apply(AnormSpec.scala:24)
    at org.specs2.specification.ContextsInjection$$anon$1$$anonfun$asResult$1.apply(ContextsInjection.scala:11)

"specs2.DefaultExecutionStrategy3" prio=10 tid=0x00007fddf83e8800 nid=0xfc1 in Object.wait() [0x00007fddf4be6000]
   java.lang.Thread.State: RUNNABLE
    at acolyte.Acolyte$.connection(Acolyte.scala:37)
    at anorm.AnormTest$class.withQueryResult(AnormSpec.scala:65)
    at anorm.AnormSpec$.withQueryResult(AnormSpec.scala:9)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$11$$anonfun$apply$12.apply(AnormSpec.scala:37)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$11$$anonfun$apply$12.apply(AnormSpec.scala:37)
    at org.specs2.specification.ContextsInjection$$anon$1$$anonfun$asResult$1.apply(ContextsInjection.scala:11)

"specs2.DefaultExecutionStrategy4" prio=10 tid=0x00007fddf8342000 nid=0xfc2 in Object.wait() [0x00007fddf4ae4000]
   java.lang.Thread.State: RUNNABLE
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
    at java.lang.Class.newInstance0(Class.java:374)
    at java.lang.Class.newInstance(Class.java:327)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:373)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
    at java.sql.DriverManager$2.run(DriverManager.java:537)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.sql.DriverManager.loadInitialDrivers(DriverManager.java:517)
    at java.sql.DriverManager.initialize(DriverManager.java:650)
    at java.sql.DriverManager.registerDriver(DriverManager.java:296)
    - locked <0x000000077bd95720> (a java.lang.Class for java.sql.DriverManager)
    at acolyte.Driver.<clinit>(Driver.java:41)
    at acolyte.Acolyte$.connection(Acolyte.scala:37)
    at anorm.AnormTest$class.withQueryResult(AnormSpec.scala:65)
    at anorm.AnormSpec$.withQueryResult(AnormSpec.scala:9)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$11$$anonfun$apply$16.apply(AnormSpec.scala:45)
    at anorm.AnormSpec$$anonfun$1$$anonfun$apply$11$$anonfun$apply$16.apply(AnormSpec.scala:45)
    at org.specs2.specification.ContextsInjection$$anon$1$$anonfun$asResult$1.apply(ContextsInjection.scala:11)
    at org.specs2.specification.ContextsInjection$$anon$1$$anonfun$asResult$1.apply(ContextsInjection.scala:11)

If you forcibly kill sbt, yes, I see an EOFException, but that's not surprising- it is being forcibly shutdown in the middle of communicating with a remove jvm.

cchantep · 2013-11-20T18:11:28Z

Do you see it ending at some point? if you don't kill?
I've let it running for more than 10min, without end.

harrah · 2013-11-20T18:28:47Z

Right, you have to kill it. But, that's a problem with the tests themselves, not sbt.

cchantep · 2013-11-20T18:29:25Z

weird

harrah · 2013-11-20T18:45:49Z

To be clear, the problem is that the tests hang. This means that the user has to kill sbt. During forcible termination, sbt generates a spurious exception that is unavoidable as far as I can see. So, I'm (re)closing this.

vaedama · 2014-05-30T04:53:10Z

@harrah I am on 0.13.0 and I am facing the same exception

Exception in thread "Thread-18" java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2577)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
at sbt.React.react(ForkTests.scala:110)
at sbt.ForkTests$$anonfun$mainTestTask$1$Acceptor$2$.run(ForkTests.scala:69)
at java.lang.Thread.run(Thread.java:695)

I am just running a simple repeat loop in a test and this is what I get before the test completes.

It's a multi-module Play project and I am on Scala 2.10.2 using ScalaTest. If I set the sbt.Keys.fork in Test := true I get till here. If I set it to false, the test silently exits without even throwing this exception.

suhinini · 2014-07-02T14:37:33Z

Gentlemen, we're still facing this, with sbt 0.12.2. Tests are succeeding, sbt is not forcibly terminated.

Exception in thread "Thread-23" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2598)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1318)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    at sbt.React.react(ForkTests.scala:98)
    at sbt.ForkTests$$anonfun$apply$2$Acceptor$2$.run(ForkTests.scala:66)
    at java.lang.Thread.run(Thread.java:745)
[info] Passed: : Total 345, Failed 0, Errors 0, Passed 341, Skipped 4
[success] Total time: 90 s, completed Jul 2, 2014 5:32:42 PM

aloiscochard · 2014-07-02T14:40:59Z

@vaedama @suhinini are you guys using scalatest?

I suspect it's an issue with that test framework.

suhinini · 2014-07-02T14:43:09Z

@aloiscochard no, we're using Specs2.

onsails · 2014-09-01T16:12:55Z

I am facing the same issue with sbt 0.13.5 and specs2. If I set fork in Test to false problem dissapears.

kapilsaini · 2015-05-12T19:20:03Z

I am still facing issue when using fork in Test := true

Can some one please suggest the fix.

velvia · 2015-05-12T23:45:31Z

It might be network related folks.

I can reproduce this test on my OSS project, http://github.com/spark-jobserver/spark-jobserver, master branch, which uses SBT 0.13.7, Scala 2.10.4, and ScalaTest, and forks tests .... IF I turn my VPN on, which screws up my localhost IP. In which case:

[warn] four warnings found
[error] Uncaught exception when running tests: java.net.ConnectException: Operation timed out
Exception in thread "Thread-4" java.io.EOFException
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2601)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1319)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
    at sbt.React.react(ForkTests.scala:114)
    at sbt.ForkTests$$anonfun$mainTestTask$1$Acceptor$2$.run(ForkTests.scala:74)
    at java.lang.Thread.run(Thread.java:745)

If I turn my VPN off, then all the tests pass.

kapilsaini · 2015-05-13T00:51:47Z

For me adding JVM memory options to build.sbt did the magic..somehow

and travis-ci/travis-ci#3775)

related to issues: sbt/sbt#653 travis-ci/travis-ci#3775

This seems to be the common fix. https://github.com/clulab/processors/pull/41/files sbt/sbt#653

Problem/Solution The Finatra TravisCI build is currently failing due to what looks like issues caused by: sbt/sbt#653 travis-ci/travis-ci#3775 Attempting the solution suggested in the latter issue and done here: nativelibs4java/scalaxy-streams@01ffbad RB_ID=901517 TBR=true

michaelahlers · 2019-02-26T14:23:22Z

Seeing this with increasing regularity, typically in CID.

harrah closed this as completed in 9702d3d Jan 29, 2013

ghost assigned harrah Feb 22, 2013

harrah added a commit that referenced this issue Mar 6, 2013

Handle exceptions not caught by the test framework when forking. Fixes …

c602233

…#653.

harrah reopened this Nov 20, 2013

harrah closed this as completed Nov 20, 2013

ianoc mentioned this issue Aug 2, 2014

Fix up the tests to run forked, and split across lots of travis builds twitter/scalding#993

Merged

ochafik added a commit to nativelibs4java/scalaxy-streams that referenced this issue Sep 30, 2015

Yay, found a workaround for EOFException w/ sudo: false (see sbt/sbt#653

01ffbad

and travis-ci/travis-ci#3775)

tjdett mentioned this issue Nov 30, 2015

JVM fork fails when using the new containter-based infrastructure travis-ci/travis-ci#3775

Closed

subotic mentioned this issue Feb 5, 2016

Continuous Integration dasch-swiss/dsp-api#16

Closed

marcovzla added a commit to clulab/processors that referenced this issue Feb 27, 2016

don't fork jvm

14b8a08

related to issues: sbt/sbt#653 travis-ci/travis-ci#3775

marcovzla mentioned this issue Feb 27, 2016

workaround for travis fork issue clulab/processors#41

Merged

aryairani pushed a commit to aryairani/dire that referenced this issue Mar 2, 2016

workaround for travis fork issue

cd50d12

This seems to be the common fix. https://github.com/clulab/processors/pull/41/files sbt/sbt#653

cheeseng mentioned this issue Mar 17, 2016

java.io.EOFException when running tests scalatest/scalatest#770

Open

schbetsy mentioned this issue Jun 16, 2017

Institutions cassandra cfpb/hmda-platform#1013

Merged

syodage mentioned this issue Oct 2, 2020

Add ApproxDistinctCount trait and two HLL++ beam extension implementations. spotify/scio#3361

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fork in test should handle exceptions during creation of test class #653

Fork in test should handle exceptions during creation of test class #653

ijuma commented Jan 25, 2013

ijuma commented Jan 25, 2013

harrah commented Jan 25, 2013

ijuma commented Jan 26, 2013

ijuma commented Jan 27, 2013

ijuma commented Jan 29, 2013

taylorleese commented Oct 29, 2013

harrah commented Nov 4, 2013

benburton commented Nov 12, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

vaedama commented May 30, 2014

suhinini commented Jul 2, 2014

aloiscochard commented Jul 2, 2014

suhinini commented Jul 2, 2014

onsails commented Sep 1, 2014

kapilsaini commented May 12, 2015

velvia commented May 12, 2015

kapilsaini commented May 13, 2015

michaelahlers commented Feb 26, 2019 •

edited

Fork in test should handle exceptions during creation of test class #653

Fork in test should handle exceptions during creation of test class #653

Comments

ijuma commented Jan 25, 2013

ijuma commented Jan 25, 2013

harrah commented Jan 25, 2013

ijuma commented Jan 26, 2013

ijuma commented Jan 27, 2013

ijuma commented Jan 29, 2013

taylorleese commented Oct 29, 2013

harrah commented Nov 4, 2013

benburton commented Nov 12, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

cchantep commented Nov 20, 2013

harrah commented Nov 20, 2013

vaedama commented May 30, 2014

suhinini commented Jul 2, 2014

aloiscochard commented Jul 2, 2014

suhinini commented Jul 2, 2014

onsails commented Sep 1, 2014

kapilsaini commented May 12, 2015

velvia commented May 12, 2015

kapilsaini commented May 13, 2015

michaelahlers commented Feb 26, 2019 • edited

michaelahlers commented Feb 26, 2019 •

edited