Loading TensorFlow after another JavaCPP preset fails with an UnsatisfiedLinkError

[petebankhead]

Description

Attempting to load TensorFlow after another JavaCPP dependency (e.g. OpenCV) fails with an UnsatisfiedLinkError. This can make it hard to use Deep Java Library + TensorFlow in an application that uses any other JavaCPP presets.

(Loading TensorFlow before OpenCV works fine)

Expected Behavior

TensorFlow can be loaded, even if other JavaCPP presets are used - irrespective of the order.

Error Message

Exception in thread "main" ai.djl.engine.EngineException: Failed to load TensorFlow native library
        at ai.djl.tensorflow.engine.TfEngine.newInstance(TfEngine.java:77)
        at ai.djl.tensorflow.engine.TfEngineProvider.getEngine(TfEngineProvider.java:40)
        at ai.djl.engine.Engine.getEngine(Engine.java:186)
        at djlTF.App.checkTensorFlow(App.java:83)
        at djlTF.App.main(App.java:38)
Caused by: java.lang.UnsatisfiedLinkError: no jnitensorflow in java.library.path: /Users/pete/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2429)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
        at java.base/java.lang.System.loadLibrary(System.java:1989)
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1825)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1416)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1227)
        at org.bytedeco.javacpp.Loader.load(Loader.java:1203)
        at org.tensorflow.internal.c_api.global.tensorflow.<clinit>(tensorflow.java:12)
        at org.tensorflow.internal.c_api.AbstractTFE_ContextOptions.newContextOptions(AbstractTFE_ContextOptions.java:41)
        at ai.djl.tensorflow.engine.javacpp.JavacppUtils.createEagerSession(JavacppUtils.java:210)
        at ai.djl.tensorflow.engine.TfEngine.newInstance(TfEngine.java:58)
        ... 4 more
Caused by: java.lang.UnsatisfiedLinkError: Could not find jnitensorflow in class, module, and library paths.
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1792)
        ... 11 more

How to Reproduce?

I've created a minimal repo that reproduces the issue, and included instructions for running it in the ReadMe:
https://github.com/petebankhead/djl-tensorflow-javacpp

What have you tried to solve it?

The problem seems to arise between LibUtils.loadLibrary and JavaCPP Loader. The former uses

System.setProperty("org.bytedeco.javacpp.platform.preloadpath", path);

But if Loader.loadProperties() has already been called, then the properties are cached and setting the system property won't have any effect.

It's possible to overcome this via reflection, using something like:

    /**
     * Reset the Loader.platformProperties private field using reflection, 
     * as this is needed to load TensorFlow after OpenCV.
     */
    private static void resetLoaderPlatformProperties() {
        Field f;
        try {
            logger.info("Resetting Loader.platformProperties using reflection");
            f = Loader.class.getDeclaredField("platformProperties");
            f.setAccessible(true);
            f.set(null, null);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

Code demonstrating that is here but I don't know if it's too messy a solution for DJL - and maybe there's a better way @saudet

[frankfliu]

@petebankhead
Thanks for dig into this issue. I think your workaround looks good to me. Do you mind raise a PR?

[saudet]

The non-hacky way of doing that would be to set the "org.bytedeco.javacpp.pathsFirst" system property to "true", which makes JavaCPP look in priority at the paths inside "java.library.path" as per System.loadLibrary(), which is used by "standard JNI", so we can add the desired paths there. That's what essentially gets done on Android where we basically have to use System.loadLibrary(). @frankfliu If that doesn't work for DJL, let's do someting about it. Please let me know why that doesn't work for you guys.

[frankfliu]

"org.bytedeco.javacpp.pathsFirst" doesn't work for us. We are actually calling Loader directly.

Here is what we did for TensorFlow:

1. We download/extract all native libraries into a cache directory for different platform
2. We set "org.bytedeco.javacpp.platform.preloadpath" properties to tell TensorFlow where to looking for all the .so files
3. The we handle over to TensorFlow to load libraries.

If Loader.loadProperties() get called before we set org.bytedeco.javacpp.platform.preloadpath, then we are in trouble. I think the hack is OK in our case. Maybe you can expose an API: Loader.loadProperties(boolean refresh) so we don't have to use reflection.

[saudet]

Right, I understand you're trying to hack JavaCPP, but I don't understand why. What doesn't work exactly when you set "org.bytedeco.javacpp.pathsFirst"? Something like the following does work and does exactly what you need, if I understand correctly your needs:

1. Download/extract all native libraries into a cache directory for different platform
2. Set "org.bytedeco.javacpp.pathsFirst" to "true" and set "java.library.path" properties to tell TensorFlow where to looking for all the .so files
3. Handle over to TensorFlow to load libraries.

[frankfliu]

For the same reason, usr_paths variable get initialized before I can set "java.library.path". Once ClassLoader is initialized, updated system properties won't take effect.

[saudet]

JavaCPP does reload "java.library.path" in "pathsFirst" mode, so that should still work.

[frankfliu]

@saudet
pathsFirst itself is cached, if Loader.loadProperties() is called before DJL code, setting org.bytedeco.javacpp.pathsFirst in DJL code takes no effect.

I tested with the follow code, it still fails to load the library:

Loader.loadProperties();
System.setProperty("org.bytedeco.javacpp.pathsFirst", "true");
System.setProperty("java.library.path", path);

[saudet]

Well, we can easily change that. Is that all that is missing?

[frankfliu]

a little bit concern about setting java.library.path, it impacts other native libraries. It that possible we set org.bytedeco.javacpp.platform.preloadpath instead of java.library.path?

[saudet]

That's not exactly what it's meant for, but if you're not having any issues with it, I guess that's fine. I've added a Loader.loadProperties(boolean forceReload) method in commit bytedeco/javacpp@232dcb0 that we can call to keep doing it like this without having to use reflection.

[saudet]

JavaCPP 1.5.9 has been released! Please upgrade to get that new Loader.loadProperties(boolean forceReload) method.