Improve deployment of native libraries #8

Open
javagl opened this Issue Aug 26, 2016 · 3 comments

Projects

None yet

3 participants

@javagl
javagl commented Aug 26, 2016 edited

(EDIT: This was supposed to be opened by jcuda - sorry, I was logged in with the wrong account)

This is a very broad issue, but it can hardly be broken down into individual steps

The current deployment of the native libraries has to be improved. Right now, the natives are deployed as real native libraries (.SO, .DLL and .DYLIB files). These are built with CMake, and declared as own artifacts with Maven, in their own POM. (See https://github.com/jcuda/jcuda/tree/master/JCudaDriverJNI for an example). The deployment could be simplified (and be more in line with the Maven philosophy) if the natives could be packed into JAR files, and unpacked and loaded at runtime. There are several (intedependent) tasks on the way to this goal:

  • It must be possible to load the natives from JAR files. The basic functionality for this has been hidden in the https://github.com/jcuda/jcuda/blob/master/JCudaJava/src/main/java/jcuda/LibUtils.java class for quite a while now, but has not officially been supported. There is an improved support for this in the JOCL LibUtils class ( https://github.com/gpu/JOCL/blob/master/src/main/java/org/jocl/LibUtils.java ), but unfortunately, these LibUtils classes have diverged. One step would be to update the LibUtils of JCuda so that it might eventually converge with the JOCL LibUtils (and maybe even become a standalone library in the future)
  • The native libraries must be versioned. When a native library is loaded from a JAR, then it is actually unpacked into the system "temp" directory, and loaded from there. Right now, the native libraries only have names like JCudaDriver-windows-86_64.dll, and thus, it would not be possible to have different versions of them on the same system. This could be solved by adding a version string, so that the library is named, for example, JCudaDriver-0.7.5c-windows-86_64.dll
  • The CMake files have to be updated to generate the natives, with version numbers, in a directory where they may be picked up by the Maven POM
  • The Maven POMs have to be updated to pack the natives into JAR files that are dedicated artifacts with appropriate classifiers, e.g. JCudaDriver-0.7.5c-windows-86_64.jar

I also opened a new thread in the forum for discussing the details, options, and the usual Maven loops that one has to jump through.

@jcuda jcuda self-assigned this Aug 26, 2016
@jcuda jcuda added a commit to jcuda/jcublas that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment 446b0a4
@jcuda jcuda added a commit to jcuda/jcuda that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment dc81548
@jcuda jcuda added a commit to jcuda/jcuda-common that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment 4ac187f
@jcuda jcuda added a commit to jcuda/jcufft that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment 059f9b8
@jcuda jcuda added a commit to jcuda/jcurand that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment d69c891
@jcuda jcuda added a commit to jcuda/jcusparse that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment 2360f3e
@jcuda jcuda added a commit to jcuda/jcusolver that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment e62ea2c
@jcuda jcuda added a commit to jcuda/jcudnn that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment 3d3bf1d
@jcuda jcuda added a commit that referenced this issue Aug 29, 2016
@jcuda Improved support for native library deployment
See #8
90713e9
@jcuda
Owner
jcuda commented Aug 29, 2016

The main tasks should be done:

  • The LibUtils class was updated (basically, made more similar to that of JOCL), so that it can load the natives from the JARs
  • The natives now carry a version number, so that they may not accidentally overwrite older natives when they are unpacked from the JARs
  • The CMake files generate the natives (with the version number) in a target directory that indicates the OS and architecture
  • The POMs have been updated...

... The last point was, once more, a hassle, and it might not yet be the final solution. Details will be discussed in the forum.

@jcuda
Owner
jcuda commented Sep 11, 2016

Sorry for the "ping", but ... I'd really appreciate feedback by @MysterionRise in view of these changes in relation to mavenized-jcuda (which (sorry!) is supposed to be made obsolete by these changes as soon as possible)

@MysterionRise

Will do my friend, should have some time tomorrow

On Sep 11, 2016 22:46, "Marco Hutter" notifications@github.com wrote:

Sorry for the "ping", but ... I'd really appreciate feedback by
@MysterionRise https://github.com/MysterionRise in view of these
changes in relation to mavenized-jcuda (which (sorry!) is supposed to be
made obsolete by these changes as soon as possible)


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#8 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA6XH2lqbXlmcfXGSMEK3hQC3liQpWrHks5qpFqugaJpZM4JuJrh
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment