Intermittent java.io.FileNotFoundException on release

[samthebest]

When running

sbt -Dpublish.url=${releases_url} clean release

we get the following stacktrace, but it works OK with sbt assembly. What is really weird is it only happens sometimes, sometimes it works!

java.io.FileNotFoundException: /var/lib/jenkins/workspace/altmetrics/target/scala-2.10/classes/com/MYCLASS$$anonfun$main$5$$anonfun$apply$3$$anonfun$apply$5.class (No such file or directory)
    at java.io.FileInputStream.open0(Native Method)
    at java.io.FileInputStream.open(FileInputStream.java:195)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at sbt.Using$$anonfun$fileInputStream$1.apply(Using.scala:73)
    at sbt.Using$$anonfun$fileInputStream$1.apply(Using.scala:73)
    at sbt.Using$$anon$2.openImpl(Using.scala:65)
    at sbt.OpenFile$class.open(Using.scala:43)
    at sbt.Using$$anon$2.open(Using.scala:64)
    at sbt.Using$$anon$2.open(Using.scala:64)
    at sbt.Using.apply(Using.scala:23)
    at sbt.IO$.transfer(IO.scala:252)
    at sbt.IO$.sbt$IO$$addFileEntry$1(IO.scala:446)
    at sbt.IO$$anonfun$sbt$IO$$writeZip$2.apply(IO.scala:454)
    at sbt.IO$$anonfun$sbt$IO$$writeZip$2.apply(IO.scala:454)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
    at sbt.IO$.sbt$IO$$writeZip(IO.scala:454)
    at sbt.IO$$anonfun$archive$1.apply(IO.scala:410)
    at sbt.IO$$anonfun$archive$1.apply(IO.scala:408)
    at sbt.IO$$anonfun$withZipOutput$1.apply(IO.scala:498)
    at sbt.IO$$anonfun$withZipOutput$1.apply(IO.scala:485)
    at sbt.Using.apply(Using.scala:24)
    at sbt.IO$.withZipOutput(IO.scala:485)
    at sbt.IO$.archive(IO.scala:408)
    at sbt.IO$.jar(IO.scala:392)
    at sbt.Package$.makeJar(Package.scala:97)
    at sbt.Package$$anonfun$3$$anonfun$apply$3.apply(Package.scala:64)
    at sbt.Package$$anonfun$3$$anonfun$apply$3.apply(Package.scala:62)
    at sbt.Tracked$$anonfun$outputChanged$1.apply(Tracked.scala:84)
    at sbt.Tracked$$anonfun$outputChanged$1.apply(Tracked.scala:79)
    at sbt.Package$.apply(Package.scala:72)
    at sbt.Defaults$$anonfun$packageTask$1.apply(Defaults.scala:685)
    at sbt.Defaults$$anonfun$packageTask$1.apply(Defaults.scala:684)
    at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:54)
    at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:53)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
[error] (compile:packageBin) java.io.FileNotFoundException: /var/lib/jenkins/workspace/altmetrics/target/scala-2.10/classes/com/MCLASS$$anonfun$main$5$$anonfun$apply$3$$anonfun$apply$5.class (No such file or directory)
[error] Total time: 57 s, completed Dec 7, 2015 4:35:59 PM

[eed3si9n]

It's hard to tell what's going on without repro steps. Maybe you should raise it up on sbt-release if there are others who's seeing the same thing.

[samthebest]

I noticed that changing the file that contained the class seems to force SBT to recompile it and it makes the error go away. It's also somewhat random which file it is. This suggests to me that SBT is doing something in parallel.

Build file:

import ReleaseTransformations._

net.virtualvoid.sbt.graph.Plugin.graphSettings

val companyName = "foo"
val domain = "com"

resolvers ++= Seq(
  some internal resolvers
)

val sparkVersion = "1.5.1"
libraryDependencies ++= Seq(
  "com.mendeley" %% "altmetrics-data-containers" % "1.0.3",
  "org.scalacheck" %% "scalacheck" % "1.12.1" % "test" withSources() withJavadoc(),
  "org.specs2" %% "specs2-core" % "2.4.15" % "test" withSources() withJavadoc(),
  "org.specs2" %% "specs2-scalacheck" % "2.4.15" % "test" withSources() withJavadoc(),

  "org.apache.spark" %% "spark-core" % sparkVersion % "provided" withSources() withJavadoc(),
  "com.amazonaws" % "aws-java-sdk" % "1.10.30" % "provided",

  "org.rogach" %% "scallop" % "0.9.5" withSources() withJavadoc(),
  "org.scalaz" %% "scalaz-core" % "7.1.4" withSources() withJavadoc(),
  "io.spray" %% "spray-json" % "1.3.2" withSources() withJavadoc(),
  "com.m3" % "curly" % "0.5.5" withSources() withJavadoc()
)

// json stuff breaks without this!
dependencyOverrides ++= Set(
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)

assemblyShadeRules in assembly := Seq(
  ShadeRule.zap("pom.xml").inAll
)

val projectName = "foo"

name := projectName

addArtifact(Artifact(projectName, "assembly"), sbtassembly.AssemblyKeys.assembly)

credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")

publishTo := Some(sys.props.get("publish.url").map(url => "altmetrics" at url)
                  .getOrElse(Resolver.file("Local ivy", Path.userHome / ".ivy2" / "local")))

releaseProcess := Seq[ReleaseStep](
  publishArtifacts // : ReleaseStep, checks whether `publishTo` is properly set up
)

scalaVersion := "2.10.4"
// stop sbt arbitrarily choosing a different scala version - use the one we know Spark is using
ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }

scalacOptions ++= Seq("-feature", "-language:reflectiveCalls")

javaOptions ++= Seq("-target", "1.8", "-source", "1.8")

organization := domain + "." + companyName

parallelExecution in Test := false

//jacoco.settings

version := "0.1.41"

plugins

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.1")

addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.0")

addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.7.5")

addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.6.0")

addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.6")

We have a maven fan join our team recently, they are going to try to rewrite our build with maven, and if they succeed we will just use maven from now on ... it seems everyone just says maven is much more stable.

[zygm0nt]

Hi, is there any update on this issue? We're experiencing this in our build. Every other deploy attempt ends with this exception.

[sebastienrainville]

FYI we had this problem as well when using sbt-assembly 0.14.0 and it's fixed now with sbt-assembly 0.14.3. Like in the build file above we are using shading and I believe that was causing the problem. It seems that it was fixed by this: sbt/sbt-assembly#197

[dwijnand]

I'm happy to go with that and close this ticket.

[zygm0nt]

I've updated sbt-assembly to 0.14.3 - and this did not solve my case. I'm still experiencing the same thing with updated version.

[ywilkof]

I've used assembly 0.14.1 with shading and exprienced a similar issue - random failure due to FileNotFoundException. I've upgraded to assembly 0.14.5 and the issue completely disappeared.

[eed3si9n]

It looks like generally the problem is related to sbt-assembly, so I am closing this here.
Hopefully the latest fixes it as some comments have mentioned.