java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF

[darkfrog26]

I have successfully been using sbt-assembly in my project for months now and recently it started complaining about duplicate entries for a couple classes after I added a couple of new dependencies. As a result I added the following to my build:

  mergeStrategy in assembly <<= (mergeStrategy in assembly) {
      case old => {
        case PathList("javax", "xml", xs @ _*) => MergeStrategy.first
        case PathList("org", "w3c", "dom", "TypeInfo.class") => MergeStrategy.first
        case x => old(x)
      }
    }

This gets me past these errors into the final building stages of my combined JAR but ultimately fails with the following:

java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
    at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:215)
    at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:109)
    at sbt.IO$.addFileEntry$1(IO.scala:441)
    at sbt.IO$$anonfun$sbt$IO$$writeZip$2.apply(IO.scala:450)
    at sbt.IO$$anonfun$sbt$IO$$writeZip$2.apply(IO.scala:450)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
    at sbt.IO$.sbt$IO$$writeZip(IO.scala:450)
    at sbt.IO$$anonfun$archive$1.apply(IO.scala:403)
    at sbt.IO$$anonfun$archive$1.apply(IO.scala:401)
    at sbt.IO$$anonfun$withZipOutput$1.apply(IO.scala:496)
    at sbt.IO$$anonfun$withZipOutput$1.apply(IO.scala:482)
    at sbt.Using.apply(Using.scala:25)
    at sbt.IO$.withZipOutput(IO.scala:482)
    at sbt.IO$.archive(IO.scala:401)
    at sbt.IO$.jar(IO.scala:384)
    at sbt.Package$.makeJar(Package.scala:107)
    at sbt.Package$$anonfun$3$$anonfun$apply$3.apply(Package.scala:72)
    at sbt.Package$$anonfun$3$$anonfun$apply$3.apply(Package.scala:70)
    at sbt.Tracked$$anonfun$outputChanged$1.apply(Tracked.scala:57)
    at sbt.Tracked$$anonfun$outputChanged$1.apply(Tracked.scala:52)
    at sbt.Package$.apply(Package.scala:80)
    at sbtassembly.Plugin$Assembly$.makeJar$1(Plugin.scala:159)
    at sbtassembly.Plugin$Assembly$.apply(Plugin.scala:175)
    at sbtassembly.Plugin$.sbtassembly$Plugin$$assemblyTask(Plugin.scala:144)
    at sbtassembly.Plugin$$anonfun$baseAssemblySettings$5.apply(Plugin.scala:332)
    at sbtassembly.Plugin$$anonfun$baseAssemblySettings$5.apply(Plugin.scala:331)
    at sbt.Scoped$$anonfun$hf9$1.apply(Structure.scala:585)
    at sbt.Scoped$$anonfun$hf9$1.apply(Structure.scala:585)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:49)
    at sbt.Scoped$Reduced$$anonfun$combine$1$$anonfun$apply$12.apply(Structure.scala:311)
    at sbt.Scoped$Reduced$$anonfun$combine$1$$anonfun$apply$12.apply(Structure.scala:311)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:41)
    at sbt.std.Transform$$anon$5.work(System.scala:71)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:232)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:232)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
    at sbt.Execute.work(Execute.scala:238)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:232)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:232)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
[error] (site/*:assembly) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF

As I understand it this shouldn't happen considering the default operation of dealing with MANIFEST.MF has worked successfully for me all along and my above additions should still be activating that. I've tried adding to my merge strategy:

case PathList("META-INF", "MANIFEST.MF") => MergeStrategy.first

But I still end up with the same error. In fact I've tried nearly every merge strategy and all fail except "discard". If I use "discard" though then the Main-Class is not set on my resulting combined JAR.

[eed3si9n]

This is very strange. Are you putting any part of your settings in configuration? Also which version of sbt-assembly, sbt are you using?

[darkfrog26]

No, nothing in configuration and using sbt-assembly: 0.8.8 and sbt 0.12.0-RC4

[eed3si9n]

I tried something like:

mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) =>
  {
    case PathList("javax", "servlet", xs @ _*) => MergeStrategy.first
    case x => old(x)
  }
}

and added META-INF/MANIFEST.MF, but saw:

[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[warn] Strategy 'discard' was applied to a file

I can't work on this without being able to reproduce this in my environment. Could you try to create a mini project that can repro this using the latest sbt 0.12.3?

[Chemmo]

Hi,

I've also come across this problem and managed to reproduce it in a trivial project. I will upload it somewhere later (cant upload to github or elsewhere from here easily).

assembly
[info] Updating {file:/C:/David/sbt/reproduce_manifest_error/}mytest...
[info] Resolving org.scala-lang#scala-library;2.10.0 ...
[info] Done updating.
[info] Compiling 1 Scala source to C:\David\sbt\reproduce_manifest_error\mytest\target\scala-2.10\classes...
[info] Packaging C:\David\sbt\reproduce_manifest_error\mytest\target\scala-2.10\mytest_2.10-1.0.jar ...
[info] Done packaging.
[info] Including scala-library.jar
[info] Including mytest_2.10-1.0.jar
[warn] Merging 'META-INF\MANIFEST.MF' with strategy 'first'
[warn] Strategy 'first' was applied to a file
[info] Packaging C:\David\sbt\reproduce_manifest_error\mytest\target\scala-2.10\mytest-assembly-1.0.jar ...
[trace] Stack trace suppressed: run last mytest/*:assembly for the full output.
error java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] Total time: 34 s, completed 17-May-2013 17:28:22

David

[eed3si9n]

@Chemmo Your output says:

[warn] Merging 'META-INF\MANIFEST.MF' with strategy 'first'

I'm guessing there's customization problem here.

[Chemmo]

What do you mean by customization problem?

I uploaded a project reproducing the issue here
http://temp-share.com/show/KdPfy1juh

David

[eed3si9n]

Here's from your build.sbt with some comments removed:

import AssemblyKeys._ 

import sbtassembly.Plugin._

import sbt.Package.ManifestAttributes

assemblySettings

name := "mytest"

test in assembly := {}

packageOptions in assembly := Seq(ManifestAttributes(("Main-Class", "test"), ("Built-By","team"), ("Implementation-Title", "console"), ("Implementation-Version", "1.0")))

mergeStrategy in assembly := { case _ => MergeStrategy.first }

You rewired mergeStrategy in assembly to blanket apply MergeStrategy.first. That's causing the problem because MANIFEST.MF needs to be discarded from all the jars.

[enijns]

I am running into the same issue. Maybe it is worth mentioning that in my case it is a multi-module sbt build?

[jgrowl]

Thanks eed3si9n! I was getting the same error because I had overridden the entire default merge strategy to take the first. I didn't realize I was wiping out the lines that discard the manifests in defaultMergeStrategy.

[malcolmgreaves]

Random internet person here: this discussion solved my problem! Thank you! (For reference, I added case PathList("META-INF", "MANIFEST.MF") => MergeStrategy.discard to my mergeStrategy function).

[kayvonr]

GOD BLESS ALL OF YOU PEOPLE FOR THIS THREAD I WAS READY TO SMASH MY COMPUTER

[schleumer]

Bless +1, "case PathList("META-INF", "MANIFEST.MF") => MergeStrategy.discard" did the trick.

[506764932]

YES case PathList("META-INF", "MANIFEST.MF") => MergeStrategy.discard trick.

[sokrahta]

always fallback to the old merge strategy, as shown in the README code sample

[emwalker]

Thank you random internet person who suggested MergeStrategy.discard.

[marcos-sb]

Thanks! Solved my problem as well.

[ariasf]

case PathList("META-INF", "MANIFEST.MF") => MergeStrategy.discard  FTW

[akonopko]

thanks people

[vikram9r]

Thank you random internet person for suggesting PathList("META-INF", "MANIFEST.MF") => MergeStrategy.discard

[sguzman]

i love this thread. 1+ from me as well since it worked

[evandertino]

Thank you guys

[eed3si9n]

If you fallback to the old strategy discarding should be there already:

assemblyMergeStrategy in assembly := {
  case PathList("javax", "servlet", xs @ _*)         => MergeStrategy.first
  // your other customization

  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}