java.lang.UnsatisfiedLinkError: no pytorch_jni in java.library.path on OS X

[erip]

I'm trying to replicate a minimal change to this demo on OS X, but I am encountering an error:

12178: Error: Uncatched exception in thread [Thread-31]
12178: Error: java.lang.UnsatisfiedLinkError: no pytorch_jni in java.library.path
12178: Error: 	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
12178: Error: 	at java.lang.Runtime.loadLibrary0(Runtime.java:870)
12178: Error: 	at java.lang.System.loadLibrary(System.java:1122)
12178: Error: 	at com.facebook.soloader.nativeloader.SystemDelegate.loadLibrary(SystemDelegate.java:22)
12178: Error: 	at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:43)
12178: Error: 	at org.pytorch.NativePeer.<clinit>(NativePeer.java:11)
12178: Error: 	at org.pytorch.Module.load(Module.java:23)

I have recompiled pytorch from source and have confirmed that libpytorch_jni is indeed in the java.library.path:

$ find $LIBTORCH_HOME/lib -name "*jni*"
/Users/erip/miniconda3/envs/tmp/lib/python3.6/site-packages/torch//lib/libfbjni.dylib
/Users/erip/miniconda3/envs/tmp/lib/python3.6/site-packages/torch//lib/libpytorch_jni.dylib

The README to this repo suggests OS X's java runtime will be stable by 1.5.0, so I'm mostly documenting this here. My pytorch version is torch==1.5.0.dev20200324.

Context: this issue

[dreiss]

Can you paste the command you are running? Did you remember to export LIBTORCH_HOME? Did you remember to export USE_LIBTORCH_NIGHTLY=1? Can you run ./gradlew --info run and paste the logging output?

[erip]

@dreiss I'm not running the exact demo but something representative with a jar build, so the export of USE_LIBTORCH_NIGHTLY isn't necessary. My exact command is:

java -jar MyApplication.jar -Djava.library.path=$LIBTORCH_HOME/lib

$LIBTORCH_HOME is exported appropriately (see above find output), and my gradle build file has the following:

dependencies {
   // ...
   compile 'org.pytorch:pytorch_java_only:1.5.0-SNAPSHOT'
}
def LIBTORCH_HOME = System.getenv('LIBTORCH_HOME')
applicationDefaultJvmArgs = ["-Djava.library.path=$LIBTORCH_HOME/lib"]

[dreiss]

What if you put -Djava.library.path=$LIBTORCH_HOME/lib before -jar on the command line?

[dreiss]

$LIBTORCH_HOME is exported appropriately (see above find output)

That find command would still work even if LIBTORCH_HOME was not exported. But your command also doesn't require it to be exported, so shrug.

[erip]

Wow, incredible debugging skills with very little info. Adding the $JAVA_OPTS before -jar seems to have worked!

[erip]

I'm a bit curious why the $JAVA_OPTS need to be present in jar invocation given the build.gradle already forces the java args in the application run, but that's likely a separate thing altogether.

[dreiss]

I think anything after -jar foo.jar is treated as an argument to your Java program. Any arguments to the JVM need to be before. Glad you got it working. Please provide feedback on the Java bindings and let us know how you're using them!

[dreiss]

I'm a bit curious why the $JAVA_OPTS need to be present in jar invocation given the build.gradle already forces the java args in the application run, but that's likely a separate thing altogether.

I think applicationDefaultJvmArgs only applies when using gradle run.

[erip]

Makes perfect sense and this seems to agree with you. Thanks for the remote rubberducking!