Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve deployment of native libraries #8

Closed
4 tasks done
javagl opened this issue Aug 26, 2016 · 4 comments
Closed
4 tasks done

Improve deployment of native libraries #8

javagl opened this issue Aug 26, 2016 · 4 comments
Assignees

Comments

@javagl
Copy link

javagl commented Aug 26, 2016

(EDIT: This was supposed to be opened by jcuda - sorry, I was logged in with the wrong account)

This is a very broad issue, but it can hardly be broken down into individual steps

The current deployment of the native libraries has to be improved. Right now, the natives are deployed as real native libraries (.SO, .DLL and .DYLIB files). These are built with CMake, and declared as own artifacts with Maven, in their own POM. (See https://github.com/jcuda/jcuda/tree/master/JCudaDriverJNI for an example). The deployment could be simplified (and be more in line with the Maven philosophy) if the natives could be packed into JAR files, and unpacked and loaded at runtime. There are several (intedependent) tasks on the way to this goal:

  • It must be possible to load the natives from JAR files. The basic functionality for this has been hidden in the https://github.com/jcuda/jcuda/blob/master/JCudaJava/src/main/java/jcuda/LibUtils.java class for quite a while now, but has not officially been supported. There is an improved support for this in the JOCL LibUtils class ( https://github.com/gpu/JOCL/blob/master/src/main/java/org/jocl/LibUtils.java ), but unfortunately, these LibUtils classes have diverged. One step would be to update the LibUtils of JCuda so that it might eventually converge with the JOCL LibUtils (and maybe even become a standalone library in the future)
  • The native libraries must be versioned. When a native library is loaded from a JAR, then it is actually unpacked into the system "temp" directory, and loaded from there. Right now, the native libraries only have names like JCudaDriver-windows-86_64.dll, and thus, it would not be possible to have different versions of them on the same system. This could be solved by adding a version string, so that the library is named, for example, JCudaDriver-0.7.5c-windows-86_64.dll
  • The CMake files have to be updated to generate the natives, with version numbers, in a directory where they may be picked up by the Maven POM
  • The Maven POMs have to be updated to pack the natives into JAR files that are dedicated artifacts with appropriate classifiers, e.g. JCudaDriver-0.7.5c-windows-86_64.jar

I also opened a new thread in the forum for discussing the details, options, and the usual Maven loops that one has to jump through.

@jcuda jcuda self-assigned this Aug 26, 2016
jcuda added a commit to jcuda/jcublas that referenced this issue Aug 29, 2016
jcuda added a commit to jcuda/jcuda that referenced this issue Aug 29, 2016
jcuda added a commit to jcuda/jcuda-common that referenced this issue Aug 29, 2016
jcuda added a commit to jcuda/jcufft that referenced this issue Aug 29, 2016
jcuda added a commit to jcuda/jcurand that referenced this issue Aug 29, 2016
jcuda added a commit to jcuda/jcusparse that referenced this issue Aug 29, 2016
jcuda added a commit to jcuda/jcusolver that referenced this issue Aug 29, 2016
jcuda added a commit to jcuda/jcudnn that referenced this issue Aug 29, 2016
jcuda added a commit that referenced this issue Aug 29, 2016
@jcuda
Copy link
Owner

jcuda commented Aug 29, 2016

The main tasks should be done:

  • The LibUtils class was updated (basically, made more similar to that of JOCL), so that it can load the natives from the JARs
  • The natives now carry a version number, so that they may not accidentally overwrite older natives when they are unpacked from the JARs
  • The CMake files generate the natives (with the version number) in a target directory that indicates the OS and architecture
  • The POMs have been updated...

... The last point was, once more, a hassle, and it might not yet be the final solution. Details will be discussed in the forum.

@jcuda
Copy link
Owner

jcuda commented Sep 11, 2016

Sorry for the "ping", but ... I'd really appreciate feedback by @MysterionRise in view of these changes in relation to mavenized-jcuda (which (sorry!) is supposed to be made obsolete by these changes as soon as possible)

@MysterionRise
Copy link

Will do my friend, should have some time tomorrow

On Sep 11, 2016 22:46, "Marco Hutter" notifications@github.com wrote:

Sorry for the "ping", but ... I'd really appreciate feedback by
@MysterionRise https://github.com/MysterionRise in view of these
changes in relation to mavenized-jcuda (which (sorry!) is supposed to be
made obsolete by these changes as soon as possible)


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#8 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA6XH2lqbXlmcfXGSMEK3hQC3liQpWrHks5qpFqugaJpZM4JuJrh
.

@jcuda
Copy link
Owner

jcuda commented Feb 20, 2017

The basic infrastructure is there and seems to work. Details about the POMs and the further path for bringing JCuda into Maven Central are discussed in #14

@jcuda jcuda closed this as completed Feb 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants