Add support for packaging native libraries in fat jars and unpacking them

[mathieuruellan]

Hello,

I'm trying to deliver a fatjar. But i'm using com.almworks.sqlite4java which depends on a native package (libsqlite4java-linux-amd64).

sqlite4java.jar and sqlite4java.so must be in the same directory.
With the requiresUnpack, i'm setting both, but only the jar is extracted in the tmp directory.

EDIT: actually, "so" dependency is never packaged in the final jar.

[wilkinsona]

How are you packaging the .so file? Is it definitely in the fat jar? Looking at the code, anything that's in lib (Boot 1.3) or BOOT-INF/lib (1.4) and marked as requiring unpacking should be unpacked. A small sample that reproduces the problem would be ideal.

[mathieuruellan]

Hi,

the lib directory (1.3.6-Release) contains only jar dependencies.

Here the pom.xml part:

            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>${springboot.version}</version>

                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                        <configuration>
                            <requiresUnpack>
                                <dependency>
                                    <groupId>com.almworks.sqlite4java</groupId>
                                    <artifactId>sqlite4java</artifactId>
                                </dependency>
                                <dependency>
                                    <groupId>com.almworks.sqlite4java</groupId>
                                    <artifactId>libsqlite4java-linux-amd64</artifactId>
                                    <type>so</type>
                                </dependency>
                            </requiresUnpack>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

here the zip:

Archive:  myscript-backend-webapp-webservices-1.0.0-Alpha1-SNAPSHOT.jar
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2016-07-05 16:05   META-INF/
      282  2016-07-05 16:05   META-INF/MANIFEST.MF
        0  2016-07-05 16:05   lib/
   129381  2012-03-01 18:53   lib/sqlite4java-0.282.jar
UNPACK:745a7e2f35fdbe6336922e0d492c979dbbfa74fb
        0  2016-07-05 16:05   com/
        0  2016-07-05 16:05   com/myscript/
        0  2016-07-05 16:05   com/myscript/backend/
        0  2016-07-05 16:05   com/myscript/backend/configuration/
        0  2016-07-05 16:05   com/myscript/backend/configuration/common/
        0  2016-07-05 16:05   com/myscript/backend/configuration/common/properties/
        0  2016-07-05 16:05   com/myscript/backend/configuration/myscript/
        0  2016-07-05 16:05   com/myscript/backend/configuration/admin/
        0  2016-07-05 16:05   assets/
        0  2016-07-05 16:05   assets/js/
        0  2016-07-05 16:05   assets/fonts/

My current workaround is: using the dependency-copy plugin (which copy well the so file), adding a manifest for the main class and the classpath, and having several jars.

[mathieuruellan]

Actually, the so files is never packages. My issue description is wrong.

[wilkinsona]

The current design is to only package anything that's a zip into lib or BOOT-INF/lib. The logic for this is in Repackager:

File file = library.getFile();
if (isZip(file)) {
    if (library.isUnpackRequired()) {
        unpackLibraries.add(library);
    }
    else {
        standardLibraries.add(library);
    }
}

I don't think there's any easy way for you to get this to work at the moment. Even if you used the assembly plugin to package the .so file and them passed that jar to Boot's repackaging, you wouldn't be able to add the comment that triggers unpacking.

[philwebb]

Unfortunately I think extended the use of fat JARs for .so files isn't something that we can focus on at the moment. The way that sqlite4java needs the .so file in the same directory is quite unique. Many other libraries bundle .so files inside the JAR and transparently unpack them as required. We'd need to make quite a lot of changes to support this one use-case, and there are other issues that have a higher priority.

I'd suggest switching to use the Maven Assembly plugin to create a more traditional .zip distribution that includes a start script.