No arm64/aarch64 support for rocksdbjni-xxx.jar

[Yikun]

We are working on the ARM64 support for Storm project, and the Storm project is using rocksdbjni to play rocksdb.

Unfortunately, there is no aarch64 support in rocksdbjni-xxx.jar, so we will get back some error like when we running the storm test in aarch64 env:

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.609 s <<< FAILURE! - in org.apache.storm.TickTupleTest
[ERROR] testTickTupleWorksWithSystemBolt Time elapsed: 0.609 s <<< ERROR!
java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni8914565631636894594.so: /tmp/librocksdbjni8914565631636894594.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64-bit .so on a AARCH64-bit platform)

Expected behavior

jar xvf rocksdbjni-6.0.1.jar | grep librocksdbjni
inflated: librocksdbjni-linux32.so
inflated: librocksdbjni-linux64.so
inflated: librocksdbjni-osx.jnilib
inflated: librocksdbjni-linux-ppc64le.so
inflated: librocksdbjni-win64.dll

Actual behavior

There is no librocksdbjni-linux-arm64.so in jar.

Steps to reproduce the behavior

jar xvf rocksdbjni-xxx.jar | grep librocksdbjni

[Yikun]

I try to build the rocksdbjni by manaully, I can build the 6.2.0 version(current master) successfully in aarch64 env after patching #5363 .

The build steps looks like:

$ export DEBUG_LEVEL=0
$ make -j8 rocksdbjava

Finally, I got the rocksdbjni-6.2.0-linux64.jar which contains the aarch64 so lib.

$ jar -xvf rocksdb/java/target/rocksdbjni-6.2.0-linux64.jar | grep lib
 inflated: librocksdbjni-linux-aarch64.so

So, does anyone know what's the gap between current and the offical arm64 rocksdbjni release? Or anybody have any plan to drive it forward?

[adamretter]

@Yikun I produce the RocksJava builds and publish them to Maven Central. Unfortunately I have no access to Arm64 hardware for either CI or releases. If there was access available to such hardware, I could also attempt to build for Arm64, as I do for ppc64le.

[Yikun]

@adamretter As I know, there are two way to have the ARM resource,

1. OpenLab can provide ARM64 vitual machine resource for open source use, you could post a "Test Request" in https://github.com/theopenlab/openlab/issues, and the team will give you a tmp vm for testing. If you have any other question you can feel free to contact me.
2. drone ci also provide ARM64 container to do some test work, like [1] I write some rocksdb build script and the results looks like [2].

So, if you need a full access for ARM64 resource you could get source according 1, and if just for testing, you could try 2.

[1] https://github.com/Yikun/arm-openlab-test/pull/18
[2] https://cloud.drone.io/Yikun/arm-openlab-test/23

[AlexanderStets]

@Yikun could you share your rocksdbjni-6.2.0-linux64.jar with the aarch64 so lib inside?

[adamretter]

@Yikun I opened theopenlab/openlab#329

[Yikun]

@AlexanderStets Sorry, the jar file size is over 10MB, I can't attach it in here. and now you can build using the master version RocksDB, the steps looks like:

1. install the java sdk and export java home
2. build the aarch64 rocksdb jni jar.

export DEBUG_LEVEL=0
make -j8 rocksdbjava

[Yikun]

@adamretter OK, the request is approved, and the resource info has been sent to you.

[adamretter]

I just released RocksDB v6.2.2 to Maven Central. It includes aarch64 support. @Yikun would you be able to test and let me know that it works for you?

[adamretter]

@Yikun @AlexanderStets btw - I also created a Docker Build Environment for building RocksJava on aarch64 - evolvedbinary/rocksjava:centos7_arm64v8-be

[Yikun]

@adamretter Great, and I saw the rocksdbjni.jar in mvn repo. But it seems does not work, when I run the storm rocksdb related test case, the exception is still raised:

"java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni9123367601253370801.so: /tmp/librocksdbjni9123367601253370801.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64-bit .so on a AARCH64-bit platform)"

And after some investigation, I found that it's still the x86-64 released:

$ readelf -h /tmp/librocksdbjni9123367601253370801.so
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 03 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  // ...
  Machine:                           Advanced Micro Devices X86-64
  // ...

The root reason is here, which was introduced in #5258 .

I guess the 6.2.2 is not cover this code, right? So, I think maybe we should add a new released after #5258 .

[adamretter]

@Yikun it looks that way, sorry. I added some improvements myself also - https://github.com/facebook/rocksdb/pull/5674/files

@gfosco could you ensure that #5258 and #5674 get into the next release please?

[Yikun]

@adamretter When will the next release coming? and do you know how to make sure the rocksdbjni works well?

Note that: We (I and @wangxiyuan) do some test on rocksdb in aarch64, like make check_some #5736 and make jtest #5751, but seems got some error. Do you know these cases are necessary for rocksdbjni or not?

[adamretter]

@gfosco were you able to make sure those two PRs mentioned above (#5258 and #5674) will be in the next release?

@Yikun I usually assume that someone has run make jtest before pushing the tag. However, I don't know that whoever does the tagging process (@gfosco ?) tests on all architectures.

[mysunshin-zc]

Excuse me，When the development of aarch64 rockdb will be updated！

[mysunshin-zc]

Excuse me，When the development of aarch64 rockdb will be updated！

[adamretter]

@gfosco @siying any update on this please?

[siying]

If there are not conflicts, I'm OK with backporting them to 6.3 and up. @gfosco any thought on it?

[siying]

@adamretter I cherry picked #5674 to both of 6.3.fb and 6.4.fb branch. But I see merge conflict when trying to merge it to 6.5.fb, 6.4.fb and 6.3.fb. Any idea why? I don't remember whether you have commit permission to those branches, but if you have it, feel free to backport them by yourself.

[adamretter]

@siying unfortunately I don't have permission to push to those branches:

remote: error: GH006: Protected branch update failed for refs/heads/6.3.fb.
remote: error: You're not authorized to push to this branch.

As far as I can tell, there is only one missing PRs now #5565... which needs to be taken into 6.3.fb.

[Yikun]

I noticed that the rocksdbjni v6.4.6 has been released, I saw all related changes has been merged.

And I try to use it in the Apache Storm project [1], the results shows that it works well in aarch64 platform.

Thanks for the efforts of the team!

[1] apache/storm#3225

[adamretter]

@Yikun you are welcome. We will also soon incorporate support for musl-libc platforms including arm-64v8.

[Myasuka]

@adamretter Could you tell me where did you build the release ARM version of RocksDB? On a ARM-based AWS?
I am also planning to build a RocksDB on arm by myself.

[adamretter]

@Myasuka We have a dedicated machine that was provided to us.