Rename duplicate files based on source jar name (feature request to build with JOGL /jogamp.org)

[GiGurra]

Jogamp JOGL (see http://jogamp.org/) is an OpenGL binding for the jvm. I have been talking to one of their maintainers about making it possible to build JOGL applications with Scala and SBT.

Using JOGL in an sbt project is mostly quite simple - it only requires you to specify two dependencies:

libraryDependencies += "org.jogamp.gluegen" % "gluegen-rt-main" % "2.2.4"
libraryDependencies += "org.jogamp.jogl" % "jogl-all-main" % "2.2.4"

This works with sbt build, package, publish, eclipse, run

However it does not work with sbt assembly. This is because JOGL/GlueGen jar files contain native dlls/so files - and these dlls have the same name (but are located in different jar files, e.g. jogl-all-2.2.4-natives-windows-i586.jar and jogl-all-2.2.4-natives-windows-amd64.jar both contain jogl_desktop.dll and a few other dlls).

The JOGL/GlueGen project have several different jar file setups that they support for dynamically finding the correct native opengl dll/so binding file inside a fat jar, as explained here:
https://jogamp.org/wiki/index.php/JogAmp_JAR_File_Handling. One of them is to create a fat jar with a directory structure inside like:
myfat.jar/natives/windows-amd64/jogl_desktop.dll

To make this work with sbt assembly, I suggest a new settings key (or extending merge strategies).
--> Make it possible to change merged file locations based on source jar name.
For example libA.jar/thelibrary.dll maps to assembly.jar/prefixBasedOnSourceJarName/thelibrary.dll

One way would be to extend the assemblyMergeStrategy API so it not only gives the source file but also what jar file it came from. This way I could easily re-map the file locations with no problem.

I posted a thread in their forums (+ had long discussion on IRC with one of their maintainers) here: http://forum.jogamp.org/Help-How-to-Build-an-executable-JAR-with-SBT-JOGL-Scala-tt4033830.html

Here is a sample sbt+JOGL project if you want to try it:
https://github.com/GiGurra/gat

• Right now this sample project is configured to only pick first/last dll files on conflict (empirically chosen ^^) - so that it at least works on 64bit windows (my platform), but this is of course a temporary test solution - I cannot deploy like this.

I hope this will not turn into a Jogamp: "you're doing it wrong" vs sbt: "No you're doing it wrong". I'm just looking for a solution for building OpenGL applications in scala and deploying them, preferably with SBT assembly like I normally do.

I haven't found any previous issues or stackoverflow topics on this topic.

[eed3si9n]

We have MergeStrategy.rename to handle LICENSE files. Not sure if that can help you out of the box.
I was able to get assembly to include native files with the following merge strategy:

assemblyMergeStrategy in assembly := {
  case PathList(ps @ _*) if ps.last endsWith ".so" => MergeStrategy.rename
  case PathList(ps @ _*) if ps.last endsWith ".dll" => MergeStrategy.rename
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

[GiGurra]

No, unfortunately that doesn't solve the issue.
It would be great if the merge strategy could somehow not only see the contained file path, but also the name of the source jar file containing it. Then I could solve it using rename merge strategy.

[eed3si9n]

I think rename uses AssemblyUtils.sourceOfFileForMerge - https://github.com/sbt/sbt-assembly/blob/0.12.0/src/main/scala/sbtassembly/AssemblyUtils.scala#L19-L33

[GiGurra]

Yes, that value is exactly what I need. Any idea if I can use that in a merge strategy?
Sry - I think this turns into me not understanding sbt or assembly well enough :/.

in pseudo code I'd like a renaming merge strategy like

if my_match_fcn(filename)
  outputFileName = createNewFileName(sourceJarName, filename)
else
  outputFileName = filename  

[eed3si9n]

Likely what you need is a custom strategy, so I'd say you can start with copy-pasting what's in rename https://github.com/sbt/sbt-assembly/blob/0.12.0/src/main/scala/sbtassembly/MergeStrategy.scala#L98-L121.

[GiGurra]

Thank you,

I guess it should be possible to just create a new anonymous scala class extending MergeStrategy in the assemblyMergeStrategy partial function case of so/dlls, like you say, just copy pasting that code even and replacing appendJarName :)

I'll try it out later and see how it goes!

I think what confused me earlier was that assemblyMergeStrategy is a partialfunction, and not the actual merge strategy itself. I'm still a little new to functional programming but loving it so far ^^.

Something like this

assemblyMergeStrategy in assembly := {
  case PathList(ps @ _*) if (ps.last endsWith ".so" || ps.last endsWith ".dll") => 
    new MergeStrategy(...) {
      ...
    }
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

[GiGurra]

Got it working :).
Thanks for all the help!

(see https://github.com/GiGurra/dcs-remote/blob/master/dcs-remote-renderer/build.sbt for implementation)

[eed3si9n]

Nicely done!

[shortwavedave]

@GiGurra could you post your build.sbt file here please? That link is dead