Skip to content

feature request: add resource merging capability #32

Closed
rkuhn opened this Issue Mar 28, 2012 · 10 comments

2 participants

@rkuhn
rkuhn commented Mar 28, 2012

Our shiny new config library relies upon reading all resources named reference.conf from the class path in order to obtain and merge the default values for all libraries using it (e.g. Akka, Play framework). sbt-assembly will include only the last such fragment from the class path into the fat jar, thereby losing all other defaults and making e.g. Akka crash upon initialization. The solution is to just concatenate those files (which can be done manually for example in a simple REPL session) and add that to the JAR, but sbt-assembly will not allow that because of a duplicate entry. Ignoring that resource will leave it out completely, so that it needs to be added manually afterwards (e.g. with zip in the shell).

A possible solution, which might also benefit other purposes, is the addition of merge capabilities to sbt-assembly. The build settings would include a transformation function Seq[File] -> Array[Byte] attached to a resource name, and then all resources with that name would be collected, transformed and the result added to the JAR.

@eed3si9n
sbt member

I am reluctant to add any sort of file transformation magic. I think sbt-assembly should focus on getting files together in one jar. You could probably define a sbt task to merge files together and generate one yourself.

@rkuhn
rkuhn commented Mar 28, 2012

Yes, I understand your reluctance, but adding an external merging task is not enough: the result would still need to be inserted into the fat jar, and that is currently not possible, given that it must have the same name as the things it was merged from.

@eed3si9n
sbt member

I wonder if I could refactor the sbt-assembly to give you a task that generates the raw list of files right before it gets packaged into the jar. Suppose we call it assembledFiles or something. That'll give you a hook point to place whatever you want in there after ignoring reference.conf from all jars.

@rkuhn
rkuhn commented Mar 28, 2012
@eed3si9n
sbt member
eed3si9n commented Apr 2, 2012

Added assembledMappings task and published 0.7.4: a2779b1

@eed3si9n eed3si9n closed this Apr 2, 2012
@rkuhn
rkuhn commented Apr 3, 2012

Thanks a bunch, will look into in when I’m back from vacation (next week)!

@rkuhn
rkuhn commented Apr 12, 2012

Sorry, it took a bit longer. I got the desired effect using the following:

assembledMappings in assembly <<= (assembledMappings in assembly) map { (old: File => Seq[(File, String)]) =>
  (f: File) => old(f) :+ (file("src/main/resources/reference.conf"), "reference.conf")
}

I guess there is some nicer way with some other SBT operator, but this also works :-)

One little thing: where did you publish 0.7.4? I had to publish-local to test this …

@eed3si9n
sbt member

It's published on the community repo:

resolvers += Resolver.url("sbt-plugin-releases",
  new URL("http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases/"))(Resolver.ivyStylePatterns)
@rkuhn
rkuhn commented Apr 12, 2012

Ah, okay. Plans for pushing that to central?

For the record, this is the nicest build.sbt code I could come up with:

excludedFiles in assembly <<= (excludedFiles in assembly) {
  (old) => (bases) => old(bases) ++ (bases flatMap (base => (base / "reference.conf").get))
}

assembledMappings in assembly <<= (assembledMappings in assembly) map {
  (old) => (f) => old(f) :+ (file("src/main/resources/reference.conf"), "reference.conf")
}

Going to write some actual merging to produce our reference.conf next.

@eed3si9n
sbt member

Plans for pushing that to central?

I don't think so. The next version of sbt would have it resolved by default and it's only sbt that needs it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.