ClassNotFoundException during readObject()

[ihji]

The following simple example works fine with scalac and scala execution but throws ClassNotFoundException in sbt run. The experimental setup is sbt 0.10 and scala 2.9.0-1.

Sample scala file:

case class A(name: String, list: List[Type])

sealed abstract class Type
case object ObjType extends Type

case class SomeMap(t: Map[A,Int]) {
  def get(a: A) : Option[Int] = { 
    t.get(a)
  }
}

object Test extends App {
  import java.io._
  val o = new ObjectOutputStream(new FileOutputStream("tmp.bin"))
  val map = SomeMap(Map(A("asdf",List(ObjType))->1))
  o.writeObject(map)
  o.close

  val i = new ObjectInputStream(new FileInputStream("tmp.bin"))
  val a = i.readObject.asInstanceOf[SomeMap]
  println("should be Some(1): "+a.get(A("asdf",List(ObjType))))
}

sbt run log:

java.lang.ClassNotFoundException: ObjType$
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:603)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1574)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at scala.collection.immutable.$colon$colon.readObject(List.scala:404)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at Test$delayedInit$body.apply(A.scala:20)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main$1.apply(App.scala:60)
    at scala.App$$anonfun$main$1.apply(App.scala:60)
    at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
    at scala.collection.immutable.List.foreach(List.scala:45)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:30)
    at scala.App$class.main(App.scala:60)
    at Test$.main(A.scala:12)
    at Test.main(A.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at sbt.Run.invokeMain(Run.scala:60)
    at sbt.Run.run0(Run.scala:53)
    at sbt.Run.execute$1(Run.scala:42)
    at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:46)
    at sbt.TrapExit$.executeMain$1(TrapExit.scala:33)
    at sbt.TrapExit$$anon$1.run(TrapExit.scala:42)

[harrah]

Not obviously a bug. Please use the mailing list for questions.

[ihji]

I found that adding fork in run := true option resolves the problem. Mentioning this kind of serialization problem in https://github.com/harrah/xsbt/wiki/Running-Project-Code would be good. Anyway, thanks for your help.

[Blaisorblade]

@ihji, I'm adding a mention of this to the Wiki, as suggested. Please take a look and feel free to improve.

[retronym]

Here's a related report on the Scala side: https://issues.scala-lang.org/browse/SI-9237 scala/bug#9237

[eed3si9n]

Ref scala/bug#9237
Ref scala/scala-dev#562 (comment)

According to scala/bug#9237 (comment), Java serialization ObjectInputStream#resolveClass specifically is incompatible with Serialization Proxy pattern and sbt's layered ClassLoader, which previously contained the issue most around List. The situation has changed in 2018 when Scala 2.13 adopted Serialization Proxy pattern for all collections. Also more prevalent use of Java serialization thanks to Spark.

I am reopening this issue.

[eed3si9n]

We will be able to work around this issue once sbt 1.3.0 comes out using #4476.
Here's how the build looks like:

ThisBuild / scalaVersion     := "2.12.8"
ThisBuild / version          := "0.1.0-SNAPSHOT"
ThisBuild / organization     := "com.example"
ThisBuild / organizationName := "example"

lazy val root = (project in file("."))
  .settings(
    name := "sbt-89",
    Compile / classLoaderLayeringStrategy := ClassLoaderLayeringStrategy.Flat,
  )

run works:

sbt:sbt-89> run
should be Some(1): Some(1)t

[success] Total time: 1 s, completed Mar 19, 2019 12:10:47 PM