8253795: Implementation of JEP 391: macOS/AArch64 Port #14

VladimirKempik · 2021-12-22T07:17:15Z

Backport of JEP-391 to jdk11u-dev
The PR has two commits, one is simple copy of linux_aarch64 to bsd_aarch64. This will allow to see the difference applied to os_cpu against linux_aarch64 version.

Total issues:
JDK-8253795: Implementation of JEP 391: macOS/AArch64 Port
JDK-8253816: Support macOS W^X
JDK-8253817: Support macOS Aarch64 ABI in Interpreter
JDK-8253818: Support macOS Aarch64 ABI for compiled wrappers
JDK-8253819: Implement os/cpu for macOS/AArch64
JDK-8253839: Update tests and JDK code for macOS/Aarch64
JDK-8254941: Implement Serviceability Agent for macOS/AArch64
JDK-8255776: Change build system for macOS/AArch64
JDK-8262903: [macos_aarch64] Thread::current() called on detached thread
JDK-8262896: [macos_aarch64] Crash in jni_fast_GetLongField

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8253795: Implementation of JEP 391: macOS/AArch64 Port ⚠️ Issue is not open.

Reviewers

Andrew Dinn (@adinn - Reviewer) ⚠️ Review applies to 774061e
Bernhard Urban-Forster (@lewurm - no project role) ⚠️ Review applies to 774061e

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/aarch64-port pull/14/head:pull/14
$ git checkout pull/14

Update a local copy of the PR:
$ git checkout pull/14
$ git pull https://git.openjdk.java.net/aarch64-port pull/14/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 14

View PR using the GUI difftool:
$ git pr show -t 14

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/aarch64-port/pull/14.diff

JDK-8253816: Support macOS W^X JDK-8253817: Support macOS Aarch64 ABI in Interpreter JDK-8253818: Support macOS Aarch64 ABI for compiled wrappers JDK-8253819: Implement os/cpu for macOS/AArch64 JDK-8253839: Update tests and JDK code for macOS/Aarch64 JDK-8254941: Implement Serviceability Agent for macOS/AArch64 JDK-8255776: Change build system for macOS/AArch64 JDK-8262903: [macos_aarch64] Thread::current() called on detached thread JDK-8262896: [macos_aarch64] Crash in jni_fast_GetLongField

bridgekeeper · 2021-12-22T07:18:38Z

👋 Welcome back vkempik! A progress list of the required criteria for merging this PR into jdk11-macos will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-12-22T07:21:22Z

⚠️ @VladimirKempik a branch with the same name as the source branch for this pull request (jdk11-macos) is present in the target repository. If you eventually integrate this pull request then the branch jdk11-macos in your personal fork will diverge once you sync your personal fork with the upstream repository.

To avoid this situation, create a new branch for your changes and reset the jdk11-macos branch. You can do this by running the following commands in a local repository for your personal fork. Note: you do not have to name the new branch NEW-BRANCH-NAME.

$ git checkout -b NEW-BRANCH-NAME
$ git branch -f jdk11-macos ec89f1b6c317932e937291f43e68d04ff77204b8
$ git push -f origin jdk11-macos

Then proceed to create a new pull request with NEW-BRANCH-NAME as the source branch and close this one.

mlbridge · 2021-12-22T07:24:45Z

Webrevs

01: Full - Incremental (9a70a86e)
00: Full (774061e9)

lewurm · 2021-12-28T09:30:51Z

I'm having troubles building it with Xcode 13.2:

=== Output from failing command(s) repeated here ===
* For target support_native_java.desktop_libsplashscreen_splashscreen_sys.o:
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:39: error: 'NSAlphaFirstBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-declarations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                      ^~~~~~~~~~~~~~~~~~~~~~~~
                                      NSBitmapFormatAlphaFirst
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:162:29: note: 'NSAlphaFirstBitmapFormat' has been expl
icitly marked deprecated here
static const NSBitmapFormat NSAlphaFirstBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaFirst", macos(10.5,10.14)) = NSBitmapFormatAlphaFirst;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:66: error: 'NSAlphaNonpremultipliedBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-decl
arations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                 NSBitmapFormatAlphaNonpremultiplied
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:163:29: note: 'NSAlphaNonpremultipliedBitmapFormat' ha
s been explicitly marked deprecated here
static const NSBitmapFormat NSAlphaNonpremultipliedBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaNonpremultiplied", macos(10.0,10.14)) = NSBitmapFormatAlphaNonpremultiplied;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:412:34: error: 'NSBorderlessWindowMask' is deprecated: first deprecated in macOS 10.12 [-Werror,-Wdeprecated-declarations]
   ... (rest of output omitted)

* All command lines available in /Users/lewurm/private/aarch64-port/build/macosx-aarch64-normal-server-release/make-support/failure-logs.
=== End of repeated output ===

Is this expected?

VladimirKempik · 2021-12-28T10:27:09Z

I'm having troubles building it with Xcode 13.2:

=== Output from failing command(s) repeated here ===
* For target support_native_java.desktop_libsplashscreen_splashscreen_sys.o:
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:39: error: 'NSAlphaFirstBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-declarations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                      ^~~~~~~~~~~~~~~~~~~~~~~~
                                      NSBitmapFormatAlphaFirst
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:162:29: note: 'NSAlphaFirstBitmapFormat' has been expl
icitly marked deprecated here
static const NSBitmapFormat NSAlphaFirstBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaFirst", macos(10.5,10.14)) = NSBitmapFormatAlphaFirst;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:66: error: 'NSAlphaNonpremultipliedBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-decl
arations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                 NSBitmapFormatAlphaNonpremultiplied
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:163:29: note: 'NSAlphaNonpremultipliedBitmapFormat' ha
s been explicitly marked deprecated here
static const NSBitmapFormat NSAlphaNonpremultipliedBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaNonpremultiplied", macos(10.0,10.14)) = NSBitmapFormatAlphaNonpremultiplied;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:412:34: error: 'NSBorderlessWindowMask' is deprecated: first deprecated in macOS 10.12 [-Werror,-Wdeprecated-declarations]
   ... (rest of output omitted)

* All command lines available in /Users/lewurm/private/aarch64-port/build/macosx-aarch64-normal-server-release/make-support/failure-logs.
=== End of repeated output ===

Is this expected?

Yes, this is expected, it's same in upstream IIRC, need to wait for a patch there first.
Just use --disable-warnings-as-errors configure argument

lewurm · 2022-01-05T14:10:56Z

Running fastdebug tier1 on macOS/AArch64 looks good:

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
>> jtreg:test/hotspot/jtreg:tier1                     1239  1238     1     0 <<
>> jtreg:test/jdk:tier1                               1872  1866     6     0 <<
   jtreg:test/langtools:tier1                         3915  3915     0     0 
   jtreg:test/nashorn:tier1                              0     0     0     0
   jtreg:test/jaxp:tier1                                 0     0     0     0
==============================
TEST FAILURE

Those 6 failures in jdk:tier also happen on Linux/AArch64, so I'll ignore them. The hotspot failure happens on test/hotspot/jtreg/runtime/ErrorHandling/TestOnError.java with the following output:

TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: '^Test Succeeded' missing from stdout

Not sure what's going on.

Circeling back to the r18 issue mentioned here: openjdk/jdk11u-dev#301 (comment)
My understanding is that the current stable release of macOS (12.1 Monterey) should already be affected by this issue, as I haven't backported https://bugs.openjdk.java.net/browse/JDK-8274795 to jdk11 yet. Do you have some better reproducer other than running Intellij? Apparently none of the tier1 tests seem to trigger it.

Edit: The release build of this branch actually runs Intellij just fine.

VladimirKempik · 2022-01-05T15:46:06Z

Circeling back to the r18 issue mentioned here: openjdk/jdk11u-dev#301 (comment) My understanding is that the current stable release of macOS (12.1 Monterey) should already be affected by this issue, as I haven't backported https://bugs.openjdk.java.net/browse/JDK-8274795 to jdk11 yet. Do you have some better reproducer other than running Intellij? Apparently none of the tier1 tests seem to trigger it.

Edit: The release build of this branch actually runs Intellij just fine.

that issue was gone after final version of r18 patch were included into jdk11u
in fact the issue was with c2 registers allocations and some #ifdef APPLE not working as expected in ad files.

ahmedmuhsin · 2022-01-05T21:23:42Z

Here are my tier2 tests results for the release build on Monterey 12.0.1/ Xcode 13.1:

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR   
   jtreg:test/hotspot/jtreg:tier2                      192   192     0     0   
>> jtreg:test/jdk:tier2                               3389  3383     4     2 <<
   jtreg:test/langtools:tier2                           12    12     0     0   
   jtreg:test/nashorn:tier2                             36    36     0     0   
   jtreg:test/jaxp:tier2                               438   438     0     0   
==============================
TEST FAILURE

Fail

java/net/NetworkInterface/UniqueMacAddressesTest.java: Tests that the MAC addresses returned by NetworkInterface.getNetworkInterfaces are unique for each adapter.
- result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: mac address uniqueness test failed
java/net/Socket/LinkLocal.java: Connecting to a link-local IPv6 address should not causes a SocketException to be thrown.
- result: Failed. Execution failed: `main' threw exception: java.net.SocketTimeoutException: Receive timed out
java/net/Socket/UrgentDataTest.java: TCP Urgent data support
- result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Exception caught (UrgentDataTest.java:120)
java/nio/channels/Selector/OutOfBand.java: OOB data causes a SocketChannel, with OOBINLINE disabled, to be selected
- result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Received more than expected

Error

java/net/ipv6tests/UdpTest.java: IPv6 support for Windows XP and 2003 server
- result: Error. Agent error: java.lang.Exception: Agent 26 timed out with a timeout of 480 seconds; check console log for any additional details
java/nio/channels/DatagramChannel/MinSendBufferSize.java: Check that new SO_SNDBUF limit on macOS is adhered to
- result: Error. test was interrupted! (timeout)

@lewurm I am also getting the same TestOnError failure in hotspot:tier1 when I run the tests on the fastdebug build. However, I do not get the 6 langtools failures. All tier1 tests pass on the release build though.

VladimirKempik · 2022-01-06T08:54:52Z

Thanks, these tier2 failures for network tests are common for macos_intel and macos_aarch64, they might requres special setup of network interfaces on the host

TestOnError failure in hotspot:tier1 when I run the tests on the fastdebug build.

I'll check that, thanks.

mlbridge · 2022-01-06T09:48:05Z

Mailing list message from Andrew Haley on aarch64-port-dev:

On 1/5/22 14:13, Bernhard Urban-Forster wrote:

My understanding is that the current stable release of macOS (12.1 Monterey) should already be affected by this issue, as I haven't backportedhttps://bugs.openjdk.java.net/browse/JDK-8274795 to jdk11 yet.

Is there an issue here that prevents 8274795 from being backported to 11? It's
obviously a bug. Are you waiting for the MacOS port?

--
Andrew Haley (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

adinn

Approved

openjdk · 2022-01-06T14:34:09Z

@VladimirKempik This change now passes all automated pre-integration checks.

After integration, the commit message for the final commit will be:

8253795: Implementation of JEP 391: macOS/AArch64 Port

Reviewed-by: adinn, burban

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the jdk11-macos branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the jdk11-macos branch, type /integrate in a new comment.

lewurm · 2022-01-07T16:13:45Z

@lewurm I am also getting the same TestOnError failure in hotspot:tier1 when I run the tests on the fastdebug build. However, I do not get the 6 langtools failures. All tier1 tests pass on the release build though.

Thanks for looking at it @ahmedmuhsin, probably an issue with my setup then.

Edit: The release build of this branch actually runs Intellij just fine.

Running on that build for >48h now. Seems good 🙂 (it doesn't just sit in the background, but I'm actually using it).

Is there an issue here that prevents 8274795 from being backported to 11? It's
obviously a bug. Are you waiting for the MacOS port?

Yes, I was hoping the macOS port nicely crashes 😉 Here is the backport: openjdk/jdk11u-dev#744

@VladimirKempik not sure how you would like to proceed with this, do you want to merge this PR into jdk11-macos or do we just let it sit and bake here? Both is fine I think.

VladimirKempik · 2022-01-08T13:26:58Z

@lewurm just proceed with openjdk/jdk11u-dev#744 it's independent of this PR

lewurm · 2022-01-10T07:49:21Z

@lewurm just proceed with openjdk/jdk11u-dev#744 it's independent of this PR

I meant this PR: #14 🙂 I think there aren't any (strict) formal processes for the aarch64 project.

VladimirKempik · 2022-01-12T11:37:18Z

About TestOnError test failure,
this is a debug-only test and it really fails on macos-aarch64 fastdebug build.
Easy to reproduce command line:

./build/macosx-aarch64-normal-server-fastdebug/images/jdk/bin/java -XX:-TransmitErrorReport -XX:-CreateCoredumpOnCrash -XX:ErrorHandlerTest=12 -XX:OnError="echo Test" -version
results in
zsh: trace trap ./build/macosx-aarch64-normal-server-fastdebug/images/jdk/bin/java
and in logs one can see:
kernel AMFI: Denying core dump for pid 69372 (java)

This happens due to the miss of this backport : openjdk/jdk@c37eabe7

we have 3 options:

backport that rather bigpatch into jdk11
exclude this test on macos-aarch64
change the test from -XX:ErrorHandlerTest=12 to -XX:ErrorHandlerTest=14 which makes it crash like expected.

@theRealAph @adinn and other, what do you think about this one ?

VladimirKempik · 2022-01-18T21:32:21Z

So what ErrorHandlerTest=12 does is basically this:
char * const dataPtr = NULL;
*dataPtr = '\0';

and this code alone ( in a simple app) does behave differently on mac_intel and mac_arm ( both with ulimit -c unlimited and user writable /cores folder)

mac_intel does this:
Segmentation fault: 11 (core dumped)
the core is then present in /cores folder

mac_arm does this:
zsh: segmentation fault ./testcrash
and printing this in dmesg:
[542308.736586]: AMFI: Denying core dump for pid 2283 (testcrash)testcrash[2283] Corpse allowed 1 of 5

so I believe the correct way is the way number 2
If no objections, I will update the PR in few days to ignore this test on mac_arm

adinn · 2022-01-21T10:58:20Z

@VladimirKempik I agree the best approach is just to exclude this on macos-aarch64.

VladimirKempik · 2022-01-22T20:40:27Z

@theRealAph Can I push this now into aarch64-port repo?

theRealAph · 2022-01-24T11:02:15Z

@theRealAph Can I push this now into aarch64-port repo?

Yes, thank you.

VladimirKempik · 2022-01-24T11:14:23Z

/integrate

openjdk · 2022-01-24T11:16:24Z

Going to push as commit 54b8298.

openjdk · 2022-01-24T11:16:40Z

@VladimirKempik Pushed as commit 54b8298.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Vladimir Kempik added 2 commits Dec 22, 2021

Simple copy of linux_aarch64 into bsd_aarch64

bda6a76

VladimirKempik mentioned this pull request Dec 22, 2021

8253795: Implementation of JEP 391: macOS/AArch64 Port openjdk/jdk11u-dev#715

Closed

3 tasks

openjdk bot added the rfr label Dec 22, 2021

adinn approved these changes Jan 6, 2022

View changes

openjdk bot added the ready label Jan 6, 2022

lewurm mentioned this pull request Jan 10, 2022

8274795: AArch64: avoid spilling and restoring r18 in macro assembler openjdk/jdk11u-dev#744

Closed

3 tasks

lewurm approved these changes Jan 10, 2022

View changes

Ignore TestOnError.java on macos_arm64

9a70a86

openjdk bot added the integrated label Jan 24, 2022

openjdk bot closed this Jan 24, 2022

openjdk bot removed ready rfr labels Jan 24, 2022

openjdk / aarch64-port Public

8253795: Implementation of JEP 391: macOS/AArch64 Port #14

8253795: Implementation of JEP 391: macOS/AArch64 Port #14

VladimirKempik commented Dec 22, 2021 •

edited by openjdk[bot] bot

bridgekeeper bot commented Dec 22, 2021

openjdk bot commented Dec 22, 2021

mlbridge bot commented Dec 22, 2021 •

edited

lewurm commented Dec 28, 2021

VladimirKempik commented Dec 28, 2021 •

edited

lewurm commented Jan 5, 2022 •

edited

VladimirKempik commented Jan 5, 2022

ahmedmuhsin commented Jan 5, 2022 •

edited by bridgekeeper[bot] bot

VladimirKempik commented Jan 6, 2022 •

edited

mlbridge bot commented Jan 6, 2022

adinn left a comment

openjdk bot commented Jan 6, 2022 •

edited

lewurm commented Jan 7, 2022 •

edited

VladimirKempik commented Jan 8, 2022

lewurm commented Jan 10, 2022

VladimirKempik commented Jan 12, 2022

VladimirKempik commented Jan 18, 2022 •

edited

adinn commented Jan 21, 2022

VladimirKempik commented Jan 22, 2022

theRealAph commented Jan 24, 2022

VladimirKempik commented Jan 24, 2022

openjdk bot commented Jan 24, 2022

openjdk bot commented Jan 24, 2022

openjdk / aarch64-port Public

8253795: Implementation of JEP 391: macOS/AArch64 Port #14

8253795: Implementation of JEP 391: macOS/AArch64 Port #14

Conversation

VladimirKempik commented Dec 22, 2021 • edited by openjdk[bot] bot

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Dec 22, 2021

openjdk bot commented Dec 22, 2021

mlbridge bot commented Dec 22, 2021 • edited

Webrevs

lewurm commented Dec 28, 2021

VladimirKempik commented Dec 28, 2021 • edited

lewurm commented Jan 5, 2022 • edited

VladimirKempik commented Jan 5, 2022

ahmedmuhsin commented Jan 5, 2022 • edited by bridgekeeper[bot] bot

VladimirKempik commented Jan 6, 2022 • edited

mlbridge bot commented Jan 6, 2022

adinn left a comment

openjdk bot commented Jan 6, 2022 • edited

lewurm commented Jan 7, 2022 • edited

VladimirKempik commented Jan 8, 2022

lewurm commented Jan 10, 2022

VladimirKempik commented Jan 12, 2022

VladimirKempik commented Jan 18, 2022 • edited

adinn commented Jan 21, 2022

VladimirKempik commented Jan 22, 2022

theRealAph commented Jan 24, 2022

VladimirKempik commented Jan 24, 2022

openjdk bot commented Jan 24, 2022

openjdk bot commented Jan 24, 2022

VladimirKempik commented Dec 22, 2021 •

edited by openjdk[bot] bot

mlbridge bot commented Dec 22, 2021 •

edited

VladimirKempik commented Dec 28, 2021 •

edited

lewurm commented Jan 5, 2022 •

edited

ahmedmuhsin commented Jan 5, 2022 •

edited by bridgekeeper[bot] bot

VladimirKempik commented Jan 6, 2022 •

edited

openjdk bot commented Jan 6, 2022 •

edited

lewurm commented Jan 7, 2022 •

edited

VladimirKempik commented Jan 18, 2022 •

edited