Failed to load any of the given libraries: [netty-tcnative-linux-aarch_64, netty-tcnative-linux-aarch_64-fedora, netty-tcnative]

[alexryan]

trying to build grpc-java on a 64 bit arm architecture.
NVIDIA 4-Plus-1™ Quad-Core ARM® Cortex™-A15 CPU

specifically the nvidia jetson tk1 dev kit.

ubuntu@tegra-ubuntu:~/grpc-java3/grpc-java$ java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

ubuntu@tegra-ubuntu:~/grpc-java3/grpc-java$ uname -a
Linux tegra-ubuntu 3.10.96-tegra #1 SMP PREEMPT Tue May 17 16:29:05 PDT 2016 aarch64 aarch64 aarch64 GNU/Linux

ubuntu@tegra-ubuntu:~/grpc-java3/grpc-java$ lsb_release -d
Description:    Ubuntu 14.04.5 LTS
ubuntu@tegra-ubuntu:~/grpc-java3/grpc-java$ 

ultimate goal is to run tensorflow in this nvidia device, but we need to be grpc-java as a part of the tensorflow installation process.

using the latest version of the source code ...
ubuntu@tegra-ubuntu:~/grpc-java3$ git clone https://github.com/grpc/grpc-java.git

building with skipCodegen=true

ubuntu@tegra-ubuntu:~/grpc-java3/grpc-java$ cat gradle.properties 
skipCodegen=true
ubuntu@tegra-ubuntu:~/grpc-java3/grpc-java$ ./gradlew build
*** Skipping the build of codegen and compilation of proto files because skipCodegen=true
...

here is the error that I am getting ...

:grpc-interop-testing:test[jetty-alpn-agent] Using: alpn-boot-8.1.8.v20160420.jar


io.grpc.testing.integration.Http2NettyTest > classMethod FAILED
    java.lang.UnsatisfiedLinkError: failed to load the required native library
        at io.netty.handler.ssl.OpenSsl.ensureAvailability(OpenSsl.java:241)
        at io.netty.handler.ssl.OpenSslContext.<init>(OpenSslContext.java:169)
        at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:347)
        at io.netty.handler.ssl.OpenSslServerContext.<init>(OpenSslServerContext.java:337)
        at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:413)
        at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:391)
        at io.grpc.testing.integration.Http2NettyTest.startServer(Http2NettyTest.java:70)

        Caused by:
        java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty-tcnative-linux-aarch_64, netty-tcnative-linux-aarch_64-fedora, netty-tcnative]
            at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:167)
            at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:325)
            at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:90)
            at io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:137)
            at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:119)
            at io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:100)
            at io.grpc.testing.integration.Http2NettyTest.startServer(Http2NettyTest.java:65)

212 tests completed, 1 failed, 4 skipped
:grpc-interop-testing:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':grpc-interop-testing:test'.

[ejona86]

There aren't pre-built binaries for tcnative on arm64. You can take a look at the Netty documentation for how to build it yourself. I don't know if anyone has built it on arm64 before, so it might be a slightly rocky ride.

If you run into trouble, I do think I know of some arm64 hardware lying around here, so it may be possible for us to try it out as well. Worst-case, you could use jetty-alpn-boot until figuring out tcnative compilation (the jetty-alpn agent has made that process much easier, but it still requires updates when the jdk version changes).

[alexryan]

This worked.
After building tcnative for arm64, grpc-java built successfully.
Thank you.

[npanpaliya]

Hi @alexryan ,
While building grpc-java on ppc64le, I'm also getting the same error as you met above. So, I wanted to know how did you fix this issue?
I built netty-tcnative on ppc64le and have its jar installed on my system in $HOME/.m2/repository. But somehow, I can't make grpc-java to use my locally built jar instead of maven central repository. Do we also need to copy this jar in $HOME/.gradle/cache/modules directory?

Any help would be appreciated.

[npanpaliya]

@ejona86 , Could you please guide me as to how I can fix this same issue on ppc64le?

I've also built and installed netty-tcnative from source. But unfortunately, grpc-java is still giving me the same error. I also tried manually copying the jar at all possible locations like in $HOME/.gradle/caches/modules_/file_/io.netty/netty-tcnative/1.1.*. But no success.

I want to know the exact path where this jar is expected to be placed.

[npanpaliya]

I got this working by temporarily putting the jar inside a new folder interop-testing/libs and then adding the reference of that folder in interop-testing/build.gradle as below -

dependencies {
 compile fileTree(dir: 'libs', include: '**/*.jar')
}

[alexryan]

@npanpaliya sorry, just saw this. I was able to solve the problem on arm64 by applying this patch while following this process.

[npanpaliya]

Hi @alexryan, Thanks and no problem!
I saw those links having patches and process. I was able to build protoc-gen-grpc-java a long ago which had served my 1st goal of building bazel and then tensorflow. But now, in order to upstream grpc-java changes for Power which is my next goal, I wanted to ensure ./gradlew build runs successfully and the main hurdle in that was netty-tcnative jar. And unfortunately, in your links, nothing is mentioned about how to use netty-tcnative jar which we built locally for our platform and that was my main problem.

I resolved that issue as mentioned in my earlier comment. Please let me know if there is any better way of achieving the same. Also, even for arm, I do not see netty-tcnative jar being published on maven repository. So, how have you get around that problem? Is netty community already working on it?