New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent java.io.FileNotFoundException on release #2301

Open
samthebest opened this Issue Dec 7, 2015 · 7 comments

Comments

Projects
None yet
6 participants
@samthebest

samthebest commented Dec 7, 2015

When running

sbt -Dpublish.url=${releases_url} clean release

we get the following stacktrace, but it works OK with sbt assembly. What is really weird is it only happens sometimes, sometimes it works!

java.io.FileNotFoundException: /var/lib/jenkins/workspace/altmetrics/target/scala-2.10/classes/com/MYCLASS$$anonfun$main$5$$anonfun$apply$3$$anonfun$apply$5.class (No such file or directory)
    at java.io.FileInputStream.open0(Native Method)
    at java.io.FileInputStream.open(FileInputStream.java:195)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at sbt.Using$$anonfun$fileInputStream$1.apply(Using.scala:73)
    at sbt.Using$$anonfun$fileInputStream$1.apply(Using.scala:73)
    at sbt.Using$$anon$2.openImpl(Using.scala:65)
    at sbt.OpenFile$class.open(Using.scala:43)
    at sbt.Using$$anon$2.open(Using.scala:64)
    at sbt.Using$$anon$2.open(Using.scala:64)
    at sbt.Using.apply(Using.scala:23)
    at sbt.IO$.transfer(IO.scala:252)
    at sbt.IO$.sbt$IO$$addFileEntry$1(IO.scala:446)
    at sbt.IO$$anonfun$sbt$IO$$writeZip$2.apply(IO.scala:454)
    at sbt.IO$$anonfun$sbt$IO$$writeZip$2.apply(IO.scala:454)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
    at sbt.IO$.sbt$IO$$writeZip(IO.scala:454)
    at sbt.IO$$anonfun$archive$1.apply(IO.scala:410)
    at sbt.IO$$anonfun$archive$1.apply(IO.scala:408)
    at sbt.IO$$anonfun$withZipOutput$1.apply(IO.scala:498)
    at sbt.IO$$anonfun$withZipOutput$1.apply(IO.scala:485)
    at sbt.Using.apply(Using.scala:24)
    at sbt.IO$.withZipOutput(IO.scala:485)
    at sbt.IO$.archive(IO.scala:408)
    at sbt.IO$.jar(IO.scala:392)
    at sbt.Package$.makeJar(Package.scala:97)
    at sbt.Package$$anonfun$3$$anonfun$apply$3.apply(Package.scala:64)
    at sbt.Package$$anonfun$3$$anonfun$apply$3.apply(Package.scala:62)
    at sbt.Tracked$$anonfun$outputChanged$1.apply(Tracked.scala:84)
    at sbt.Tracked$$anonfun$outputChanged$1.apply(Tracked.scala:79)
    at sbt.Package$.apply(Package.scala:72)
    at sbt.Defaults$$anonfun$packageTask$1.apply(Defaults.scala:685)
    at sbt.Defaults$$anonfun$packageTask$1.apply(Defaults.scala:684)
    at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:54)
    at scala.Function2$$anonfun$tupled$1.apply(Function2.scala:53)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
[error] (compile:packageBin) java.io.FileNotFoundException: /var/lib/jenkins/workspace/altmetrics/target/scala-2.10/classes/com/MCLASS$$anonfun$main$5$$anonfun$apply$3$$anonfun$apply$5.class (No such file or directory)
[error] Total time: 57 s, completed Dec 7, 2015 4:35:59 PM
@eed3si9n

This comment has been minimized.

Member

eed3si9n commented Dec 8, 2015

It's hard to tell what's going on without repro steps. Maybe you should raise it up on sbt-release if there are others who's seeing the same thing.

@samthebest

This comment has been minimized.

samthebest commented Dec 8, 2015

I noticed that changing the file that contained the class seems to force SBT to recompile it and it makes the error go away. It's also somewhat random which file it is. This suggests to me that SBT is doing something in parallel.

Build file:

import ReleaseTransformations._

net.virtualvoid.sbt.graph.Plugin.graphSettings

val companyName = "foo"
val domain = "com"

resolvers ++= Seq(
  some internal resolvers
)

val sparkVersion = "1.5.1"
libraryDependencies ++= Seq(
  "com.mendeley" %% "altmetrics-data-containers" % "1.0.3",
  "org.scalacheck" %% "scalacheck" % "1.12.1" % "test" withSources() withJavadoc(),
  "org.specs2" %% "specs2-core" % "2.4.15" % "test" withSources() withJavadoc(),
  "org.specs2" %% "specs2-scalacheck" % "2.4.15" % "test" withSources() withJavadoc(),

  "org.apache.spark" %% "spark-core" % sparkVersion % "provided" withSources() withJavadoc(),
  "com.amazonaws" % "aws-java-sdk" % "1.10.30" % "provided",

  "org.rogach" %% "scallop" % "0.9.5" withSources() withJavadoc(),
  "org.scalaz" %% "scalaz-core" % "7.1.4" withSources() withJavadoc(),
  "io.spray" %% "spray-json" % "1.3.2" withSources() withJavadoc(),
  "com.m3" % "curly" % "0.5.5" withSources() withJavadoc()
)

// json stuff breaks without this!
dependencyOverrides ++= Set(
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)

assemblyShadeRules in assembly := Seq(
  ShadeRule.zap("pom.xml").inAll
)

val projectName = "foo"

name := projectName

addArtifact(Artifact(projectName, "assembly"), sbtassembly.AssemblyKeys.assembly)

credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")

publishTo := Some(sys.props.get("publish.url").map(url => "altmetrics" at url)
                  .getOrElse(Resolver.file("Local ivy", Path.userHome / ".ivy2" / "local")))

releaseProcess := Seq[ReleaseStep](
  publishArtifacts // : ReleaseStep, checks whether `publishTo` is properly set up
)

scalaVersion := "2.10.4"
// stop sbt arbitrarily choosing a different scala version - use the one we know Spark is using
ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }

scalacOptions ++= Seq("-feature", "-language:reflectiveCalls")

javaOptions ++= Seq("-target", "1.8", "-source", "1.8")

organization := domain + "." + companyName

parallelExecution in Test := false

//jacoco.settings

version := "0.1.41"

plugins

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.1")

addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.0")

addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.7.5")

addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.6.0")

addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.6")

We have a maven fan join our team recently, they are going to try to rewrite our build with maven, and if they succeed we will just use maven from now on ... it seems everyone just says maven is much more stable.

@zygm0nt

This comment has been minimized.

zygm0nt commented May 31, 2016

Hi, is there any update on this issue? We're experiencing this in our build. Every other deploy attempt ends with this exception.

@sebastienrainville

This comment has been minimized.

sebastienrainville commented Jul 1, 2016

FYI we had this problem as well when using sbt-assembly 0.14.0 and it's fixed now with sbt-assembly 0.14.3. Like in the build file above we are using shading and I believe that was causing the problem. It seems that it was fixed by this: sbt/sbt-assembly#197

@dwijnand

This comment has been minimized.

Member

dwijnand commented Jul 1, 2016

I'm happy to go with that and close this ticket.

@zygm0nt

This comment has been minimized.

zygm0nt commented Jul 4, 2016

I've updated sbt-assembly to 0.14.3 - and this did not solve my case. I'm still experiencing the same thing with updated version.

@ywilkof

This comment has been minimized.

ywilkof commented Aug 10, 2017

I've used assembly 0.14.1 with shading and exprienced a similar issue - random failure due to FileNotFoundException. I've upgraded to assembly 0.14.5 and the issue completely disappeared.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment