Enable building jna for Darwin arm64 #1238
Conversation
native/Makefile
Outdated
| # JAWT linkage handled by -framework JavaVM | ||
| LIBS= | ||
| LIBS+=-framework JavaVM | ||
| endif |
There was a problem hiding this comment.
Is linking against JavaVM not required anymore?
There was a problem hiding this comment.
Apple does not ship a JavaVM framework anymore starting from Big Sur.
There was a problem hiding this comment.
So do we need the JavaVM framework? Or could we drop it from the build entirely?
There was a problem hiding this comment.
It seems to me this was only required for JAWT. Looks to me like it could be dropped for non-arm64 as well.
|
It is possible to build the Zero variant yourself, if you have access to the hardware: https://github.com/apple/openjdk/ |
|
All my questions might have hinted, that I don't own Apple hardware, and I don't see me owning any. For the test error in the callback test, you can try to narrow it down by adding printfs in the testlibrary and/or the native dispatch library to see when it fails or how long it works. |
|
I can confirm this is building a universal |
|
So we need a strategy how to proceed: If I remember correctly, Apple does like fat binaries only very selectively. In principle it should be possible to build a fat binary, that supports x86, x86_64 and arm64. This will need support from XCode to build all three binaries. The alternative would be to build the darwin-x86, darwin-x86-64, darwin-aarch64 separately, and merge only at build time. We could then also provide a way to drop the darwin special handling (fat binaries). We could bundle the mentioned individual darwin libraries and the fat binary. What I notice in the invocation above is the usage of the SDK libffi - can we build from the embedded libffi source/can we update it? |
@matthiasblaesing this is the strategy I've used in another aarch64 PR (2bf1593 in #1264). Outdated, click to expand
Edit: Sorry, I had missed the native/README.libffi which has explicit instructions for bumping the subtree. According to conversation in #1259, there's divergence issues so I'm still a bit confused as to how best to tackle this. |
It appears XCode 10 dropped 32-bit support, so the following is not possible using XCode 12. XCode 12 is required for arm64 support. clang \
-arch i386 \
-arch x86_64 \
-arch arm64 \
hello.c -o hello
# ld: warning: missing required architecture i386 in file [...]/MacOSX.sdk/usr/lib/libSystem.tbd (3 slices)
# Undefined symbols for architecture i386:XCode 12 can successfully build both
Edit: Just realized Lines 436 to 444 in 816fdd2 Hidden, click to expand
clang -arch x86_64 hello.c -o hello_x86_64
file hello_x86_64
# hello_x86_64 (for architecture x86_64): Mach-O 64-bit executable x86_64
# ^--- 64-bit Intel. Good, expected.
clang -arch arm64 hello.c -o hello_aarch64
file hello_aarch64
# hello_aarch64 (for architecture arm64): Mach-O 64-bit executable arm64
# ^--- 64-bit ARM. Good, expected.
lipo hello_x86_64 hello_aarch64 -create -output hello
file hello
# hello: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
# hello (for architecture x86_64): Mach-O 64-bit executable x86_64
# hello (for architecture arm64): Mach-O 64-bit executable arm64
# ^--- Successfully combined. |
|
So the option would be: We build the architecture dependent libraries individually on a matching XCode version and update In case of an incompatible api change, the whole file will be cleared, rebuilds for mac os need to be build incrementally. |
The embedded libffi compiles fine, but the resulting binary does not work correctly for me. Until libffi is updated, using the system libffi seems to be the best option.
Seems to me, the easiest way would to split off a What do you think about this? |
fkistner
force-pushed
the
darwin_arm64
branch
from
32c8915
to
16685bd
Compare
|
|
|
Added |
Please see if rebasing this work onto the work done in #1264 helps. That branch holds upstream libffi. I have some hope, that apple did the sane thing and upstreamed its changes. For the system libffi: It is dynamically linked or statically build into the jnidispatch library? What are the licensing terms? JNA already allows linking against a system libffi, it is the mode, that is used in the debian builds. However, that will utterly fail on non apple platforms (though I don't now if there are any serious darwin users outside apple).
I would not do it right now. For me this is either-or,
In the long run we should see, that the CI infrastructure gets a branch, that is executed on XCode 12, so the binary can be build from there. What I envision is this (only for apple, only while fatbinaries are targetted).
We end up with a darwin binary, that supports all three architectures. So could you merge the darwin-aarch64 binary into the darwin binary? |
Hi, @Vzor- and myself are working on a modification of this PR that does each arch separately, using I think this is slightly more intuitive than the proposal above which:
Instead we're focusing on this:
Will share the code soon, probably as a GitHub compare link. |
Just an update on this comment, since this was posted, Zulu and Microsoft (yes, Microsoft 😄)both have Apple Silicon builds available. At the time of writing this, Microsoft's is broken, so Azul's is the only precompiled version. Microsoft has informed me they'll have a new build out soon. |
|
Let me share a few command from real machine with M1 and macOS 12: It is called itself |
|
@tresf Microsoft's build doesn't work. It crashed on start. :( Zulu more stable, but it is |
OpenJDK as well as JNA call it |
|
@tresf indeed. |
I've already posted this information and a link to the details. |
Arm64 calling convention requires varargs to always be passed on the stack. JNA's and ffi's argument handling logic seems to handle this case correctly, but it is important that the Java definitions match their native counterparts exactly.
Big Sur uses a shared dylib cache to lookup system libraries and frameworks. Checking existence in the file system will fail, but loading them will succeed nevertheless.
fkistner
force-pushed
the
darwin_arm64
branch
4 times, most recently
from
b802350
to
9e6fd33
Compare
fkistner
force-pushed
the
darwin_arm64
branch
from
9e6fd33
to
536914a
Compare
|
Merged as #1297. |
…es tests using testcontainers on apple mac M1 The 5.6.0 version of jna doesn't work on apple mac m1, this is fixed in 5.7.0, but this commit suggests upgrading to the latest version, which currently is 5.13.0 jna has no transitive dependencies and jna release notes does not mention any problems upgrading to 5.13.0 for more info: https://github.com/java-native-access/jna/blob/master/CHANGES.md java-native-access/jna#1238 https://www.testcontainers.org/
We are interested in getting JNA running on Darwin arm64.
I was able to get JNA compiling and working for the use cases most important to us with these changes and
env JDK_HOME=… SDKROOT=… ant -DCC=clang -Dmake.OPTS="DYNAMIC_LIBFFI=true DARWIN_USE_SDK_LIBFFI=true".Nevertheless, I am not too familiar with JNA and would love to hear your feedback and whether you are interesting in the changes in their current state.
Summary of the changes:
libffi.Adapted
LibraryLoadTestaccordingly.Adapted
VarArgsTestaccordingly.Unfortunately, some tests still fail…