Add support for Apple Silicon to RocksJava

[etherond]

Fixes #7720

Updated Makefile with flags to define target architecture when compiling/linking,
and added goal rocksdbjavastaticosxub to build a OS X Universal Binary native library.

[pdillinger]

Thanks! Would you be up for adding a CircleCI build (.circleci/config.yaml) for building the universal binary target? (I'm assuming you can build arm on x86, and assuming we only have x86 Macs available in CircleCI.)

[adamretter]

@etherond Thanks for this. Before we merge it, I just want to understand how this effects the current builds that we produce for RocksJava releases. I am a bit concerned about removing the existing calls to rocksdbjavastatic in favour of rocksdbjavasticosxub.

I am not opposed to having a Universal Binary target. However for releases I would ideally like to build by default separate dedicated macOS libs for x86 and Arm M1 and load them depending on the platform - just as we do for different Linux architectures. Does that sound reasonable?

[Dharin-shah]

can we please have this merged :), would love to see some performance out of intelij, java on arm :P

[etherond]

Hey guys, sorry it took some time but here is the PR updated with your feedback:

• Added CircleCI job to build universal binary target (now renamed to rocksdbjavastaticosx_ub), following @pdillinger's comment
• Added target rocksdbjavastaticosx to build separate x86 and ARM libraries and load them depending on the platform, following @adamretter's comment

A few things to note:

• CircleCI doesn't have ARM Mac executors, therefore I currently don't see any way to run tests against the ARM library on CircleCI
• Building an ARM library on a x86 machine is possible but my different attempts seemed to indicate that at least Xcode >= 12.5.1 is required, so I updated the job configuration to use the appropriate executor. This raised some errors during the build, that could be fixed by upgrading the Python version and using JDK 8 (the Xcode 12.5.1 executor comes with JDK 11 by default)
• Compilation/linking depends on the jemalloc library, which causes an error when building the ARM library on CircleCI since the jemalloc library found on the system is x86 and not ARM. I imagine a way to go around this would be to compile jemalloc from sources as part of the RocksDB build process, as what's done for other dependencies, such as Bzip2. For now, I simply disabled the use of jemalloc when targeting ARM Mac.

[garyschulte]

This sure would be a nice Christmas present to all of us M1 folks. Just sayin. 🎄

[adamretter]

@pdillinger Unfortunately this has broken the ability to create release builds of RocksJava.

The release process used to be as simple as running PORTABLE=1 ROCKSDB_DISABLE_JEMALLOC=true make rocksdbjavastaticreleasedocker on a macOS machine.

Unfortunately there are now no Jar files generated for macOS and there are no javadoc or sources jar files generated either. I am investigating...

[adamretter]

First problem I have is that ROCKSDB_DISABLE_JEMALLOC=true make -j6 rocksdbjavastaticosx_ub fails.
I also get the same failure when running ROCKSDB_DISABLE_JEMALLOC=true make -j6 rocksdbjavastaticosx!

The x86_64 bit works as before, but when you reach the cross-compiling of the M1 binaries then you get the errors below.

• XCode 13.2.1
• macOS Monterey (12.2.1)
• MacBook Pro (15-inch, 2019)
• Processor 2.4 GHz 8-Core Intel Core i9

❯ clang++ --version
Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: x86_64-apple-darwin21.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Relevant Make output is:

ARCHFLAG="-arch arm64" /Applications/Xcode.app/Contents/Developer/usr/bin/make rocksdbjavastatic_libobjects
...

  CC       jls/table/block_based/full_filter_block.o
In file included from table/block_based/filter_policy.cc:26:
In file included from ./util/bloom_impl.h:21:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/immintrin.h:13:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/x86gprintrin.h:15:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/hresetintrin.h:42:27: error: invalid input constraint 'a' in asm
  __asm__ ("hreset $0" :: "a"(__eax));
                          ^
In file included from table/block_based/filter_policy.cc:26:
In file included from ./util/bloom_impl.h:21:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/immintrin.h:17:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:33:5: error: use of undeclared identifier '__builtin_ia32_emms'; did you mean '__builtin_isless'?
    __builtin_ia32_emms();
    ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/math.h:649:12: note: '__builtin_isless' declared here
    return isless(__lcpp_x, __lcpp_y);
           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/math.h:545:22: note: expanded from macro 'isless'
#define isless(x, y) __builtin_isless((x),(y))
                     ^
In file included from table/block_based/filter_policy.cc:26:
In file included from ./util/bloom_impl.h:21:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/immintrin.h:17:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:33:25: error: too few arguments to function call, expected 2, have 0
    __builtin_ia32_emms();
    ~~~~~~~~~~~~~~~~~~~~^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:50:19: error: use of undeclared identifier '__builtin_ia32_vec_init_v2si'
    return (__m64)__builtin_ia32_vec_init_v2si(__i, 0);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:67:12: error: use of undeclared identifier '__builtin_ia32_vec_ext_v2si'
    return __builtin_ia32_vec_ext_v2si((__v2si)__m, 0);
           ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:129:19: error: use of undeclared identifier '__builtin_ia32_packsswb'
    return (__m64)__builtin_ia32_packsswb((__v4hi)__m1, (__v4hi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:159:19: error: use of undeclared identifier '__builtin_ia32_packssdw'
    return (__m64)__builtin_ia32_packssdw((__v2si)__m1, (__v2si)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:189:19: error: use of undeclared identifier '__builtin_ia32_packuswb'
    return (__m64)__builtin_ia32_packuswb((__v4hi)__m1, (__v4hi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:216:19: error: use of undeclared identifier '__builtin_ia32_punpckhbw'
    return (__m64)__builtin_ia32_punpckhbw((__v8qi)__m1, (__v8qi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:239:19: error: use of undeclared identifier '__builtin_ia32_punpckhwd'
    return (__m64)__builtin_ia32_punpckhwd((__v4hi)__m1, (__v4hi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:260:19: error: use of undeclared identifier '__builtin_ia32_punpckhdq'
    return (__m64)__builtin_ia32_punpckhdq((__v2si)__m1, (__v2si)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:287:19: error: use of undeclared identifier '__builtin_ia32_punpcklbw'
    return (__m64)__builtin_ia32_punpcklbw((__v8qi)__m1, (__v8qi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:310:19: error: use of undeclared identifier '__builtin_ia32_punpcklwd'
    return (__m64)__builtin_ia32_punpcklwd((__v4hi)__m1, (__v4hi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:331:19: error: use of undeclared identifier '__builtin_ia32_punpckldq'
    return (__m64)__builtin_ia32_punpckldq((__v2si)__m1, (__v2si)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:352:19: error: use of undeclared identifier '__builtin_ia32_paddb'; did you mean '__builtin_arm_addg'?
    return (__m64)__builtin_ia32_paddb((__v8qi)__m1, (__v8qi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:352:19: note: '__builtin_arm_addg' declared here
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:352:19: error: first argument of MTE builtin function must be a pointer ('__v8qi' (vector of 8 'char' values) invalid)
    return (__m64)__builtin_ia32_paddb((__v8qi)__m1, (__v8qi)__m2);
                  ^                    ~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:373:19: error: use of undeclared identifier '__builtin_ia32_paddw'; did you mean '__builtin_arm_addg'?
    return (__m64)__builtin_ia32_paddw((__v4hi)__m1, (__v4hi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:352:19: note: '__builtin_arm_addg' declared here
    return (__m64)__builtin_ia32_paddb((__v8qi)__m1, (__v8qi)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:373:19: error: first argument of MTE builtin function must be a pointer ('__v4hi' (vector of 4 'short' values) invalid)
    return (__m64)__builtin_ia32_paddw((__v4hi)__m1, (__v4hi)__m2);
                  ^                    ~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:394:19: error: use of undeclared identifier '__builtin_ia32_paddd'; did you mean '__builtin_arm_addg'?
    return (__m64)__builtin_ia32_paddd((__v2si)__m1, (__v2si)__m2);
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/mmintrin.h:352:19: note: '__builtin_arm_addg' declared here
    return (__m64)__builtin_ia32_paddb((__v8qi)__m1, (__v8qi)__m2);
                  ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
  CC       jls/table/block_based/hash_index_reader.o
20 errors generated.
make[2]: *** [jls/table/block_based/filter_policy.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [rocksdbjavastaticosx_arch_arm64] Error 2
make: *** [rocksdbjavastaticosx_archs] Error 2

At a bit of a guess, it looks to me that:

#ifdef HAVE_AVX2
#include <immintrin.h>
#endif

is activated in util/bloom_impl.h - however as far as I am aware, M1 CPU does not have AVX2...

---

Whilst this appears to be passing in CI - https://app.circleci.com/pipelines/github/facebook/rocksdb/12065/workflows/1072da36-c82f-46c1-ba16-a8b067a5d0ca/jobs/316046?invite=true#step-108-1418

This is because CI does not have AVX2 support, and so this compilation code path is not tested in CI.
I will try and figure out how to remove HAVE_AVX2 from PLATFORM_CCFLAGS and PLATFORM_CXXFLAGS when cross compiling for M1...