"Duplicated entry" error when I run assembly twice

[mateiz]

I recently set up Spark (http://github.com/mesos/spark) to use the sbt-assembly plugin, and it works great the first time I run sbt assembly, but if I run this again afterwards I get errors like this:

[error] {file:/Users/matei/workspace/spark/}core/*:assembly: java.util.zip.ZipException: duplicate entry: spark/Partitioner.class

If I do sbt clean followed by assembly again though, it works fine. Any idea why this is happening? Am I doing something wrong in my project? Here is my SBT script: https://github.com/mesos/spark/blob/master/project/SparkBuild.scala.

[eed3si9n]

The way sbt-assembly is implemented is that it unzips all the jars in the classpath and add *.class to the assembled jar in addition to the regular *.class files. You have the following entry there:

unmanagedJars in Compile <<= baseDirectory map { base => (base ** "*.jar").classpath }

Could it be picking up other stuff like the previously assembled jar in target/?

[mateiz]

Thanks, that was the problem! I changed it to only add the JARs in lib.