Fix regression when using Jar's to force proper manifest files #102
Comments
|
I only use sbt occasionally and am not familiar with it's internals any longer. At the time I send the PRs, I was in need for a solution for #60 but lacked the detailed understanding (I think I made that clear in my communications at that time and was even searching for help in the chats). I ended up with #61 and #64, which worked for us but, as it turned out, was not the best solution. I tried to come up with a executable integration test case, which is IMHO the key to a proper plugin implementation. Try to reproduce any bugs as test cases and make them succeed. That being said, I'm sorry that I can't help with any sbt specifics. I can tell that bnd (the OSGI bundle creator tool) needs to see the OSGi Bundles of the dependencies. Just feeding it the classfiles without the proper OSGi Manifests is not enough. Only the manifests contain the exported package versions, which are needed to calculate proper import package version constraints. The essential change I tried to make in PR #64 was to replace some directories with classfiles (the dependencies) with their JARs containing proper OSGi manifests. TBH, my confidence, that this issue can be solved in a clean way is rather small. My solution failed in a very race-y way. There seems to be no API in sbt, that ensures that the result of tasks, once they are done, stays stable. I have no idea (and also did not receive any helpful idea from others) how this can be fixed. My advive for any dev who targets OSGi, don't do it with sbt. My OSGi projects went a lot smother since they switched over to Mill. Don't get me wrong. I don't want to discredit sbt or sbt-osgi. It's a very capable build tool driving many successful projects. But there is no intersection with OSGi knowledge anymore. Given how long the issue is known and there is no clear idea how to fix it, it's unlikely that a fix is easy and prompt. And to leave with a more constructive note. Be prepared to deep dive into sbt internals yourself. Maybe a working solution needs a bit out-of-the-box thinking, trading performance for correctness. The essential task is pretty easy. Just get the jars of your dependencies and feed it to bnd. TBH, I don't know why it is so damn hard to get a grab on these artifacts trough the sbt dependency graph. There must be a simple solution. |
|
So I think I have a little bit of a better understanding of whats going on. Specifically in the case of pekko, we are dealing with a farjar from sbt-assembly which by definition doesn't really have dependencies (hence why its not included in This means that at least for this issue specifically, its not to do with the racey issues with sbt as you describe but rather the distinction between jars that have dependencies vs jars that don't (which does raise other questions regarding how should treat sbt-osgi when used by other plugins like sbt-assembly). With that being said this does raise issues regarding knowing if the result of an sbt task is "dirty" or not but there isn't much that I can do here (maybe @eed3si9n can comment), moving to mill at this point for Pekko is out of the question. |
So I did some digging to figure out the root cause of apache/pekko#757 and I managed to bisect the change which caused the regression which happens to be #64 . When using
at https://github.com/sbt/sbt-osgi/blob/main/src/main/scala/com/typesafe/sbt/osgi/Osgi.scala#L186 and
sbt protobuf-v3/osgiBundlewithin pekko to diagnose the content that is being packaged, without #64 you getwhere as with #64 you get
As you can see the change means that all of the
.classfiles end up getting stripped out of the osgi-bundle.If we do some more digging we can see what is causing the regression, if I you do
show protobuf-v3/Compile/fullClasspathwhich is what was used without #64 you getwhere as if you do
show protobuf-v3/Compile/dependencyClasspathAsJars(which is with #64) you getThe
Attributed(/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/stripped/stripped/pekko-protobuf-v3-assembly-1.1.0-M0-1-SNAPSHOT.jar)happens to be missing.This is very likely a result of the bespoke way that we generate the jar for protobuf-v3 (see https://github.com/apache/incubator-pekko/blob/main/build.sbt#L359-L387)
@lefou Since you implemented #64 you know what the underlying motivation for the change. At least on a glance to me it seems like a quick way to fix this is to combine both
(dependencyClasspathAsJars in Compile).value.map(_.data) ++ (products in Compile).valueand(fullClasspath in Compile)but use(dependencyClasspathAsJars in Compile).value.map(_.data) ++ (products in Compile).valueas a preference which should still solve the underlying problem described in #60 .@raboof Maybe you can provide some underlying context here regarding the implementation of the protobuf-v3 sbt module.
@romainreuillon If you have time to look into this that would also be great.
The text was updated successfully, but these errors were encountered: