Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8253795: Implementation of JEP 391: macOS/AArch64 Port #14

Closed
wants to merge 3 commits into from
Closed

8253795: Implementation of JEP 391: macOS/AArch64 Port #14

wants to merge 3 commits into from

Conversation

VladimirKempik
Copy link

@VladimirKempik VladimirKempik commented Dec 22, 2021

Backport of JEP-391 to jdk11u-dev
The PR has two commits, one is simple copy of linux_aarch64 to bsd_aarch64. This will allow to see the difference applied to os_cpu against linux_aarch64 version.

Total issues:
JDK-8253795: Implementation of JEP 391: macOS/AArch64 Port
JDK-8253816: Support macOS W^X
JDK-8253817: Support macOS Aarch64 ABI in Interpreter
JDK-8253818: Support macOS Aarch64 ABI for compiled wrappers
JDK-8253819: Implement os/cpu for macOS/AArch64
JDK-8253839: Update tests and JDK code for macOS/Aarch64
JDK-8254941: Implement Serviceability Agent for macOS/AArch64
JDK-8255776: Change build system for macOS/AArch64
JDK-8262903: [macos_aarch64] Thread::current() called on detached thread
JDK-8262896: [macos_aarch64] Crash in jni_fast_GetLongField


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8253795: Implementation of JEP 391: macOS/AArch64 Port ⚠️ Issue is not open.

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/aarch64-port pull/14/head:pull/14
$ git checkout pull/14

Update a local copy of the PR:
$ git checkout pull/14
$ git pull https://git.openjdk.java.net/aarch64-port pull/14/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 14

View PR using the GUI difftool:
$ git pr show -t 14

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/aarch64-port/pull/14.diff

Vladimir Kempik added 2 commits Dec 22, 2021
JDK-8253816: Support macOS W^X
JDK-8253817: Support macOS Aarch64 ABI in Interpreter
JDK-8253818: Support macOS Aarch64 ABI for compiled wrappers
JDK-8253819: Implement os/cpu for macOS/AArch64
JDK-8253839: Update tests and JDK code for macOS/Aarch64
JDK-8254941: Implement Serviceability Agent for macOS/AArch64
JDK-8255776: Change build system for macOS/AArch64
JDK-8262903: [macos_aarch64] Thread::current() called on detached thread
JDK-8262896: [macos_aarch64] Crash in jni_fast_GetLongField
@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Dec 22, 2021

👋 Welcome back vkempik! A progress list of the required criteria for merging this PR into jdk11-macos will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Dec 22, 2021

⚠️ @VladimirKempik a branch with the same name as the source branch for this pull request (jdk11-macos) is present in the target repository. If you eventually integrate this pull request then the branch jdk11-macos in your personal fork will diverge once you sync your personal fork with the upstream repository.

To avoid this situation, create a new branch for your changes and reset the jdk11-macos branch. You can do this by running the following commands in a local repository for your personal fork. Note: you do not have to name the new branch NEW-BRANCH-NAME.

$ git checkout -b NEW-BRANCH-NAME
$ git branch -f jdk11-macos ec89f1b6c317932e937291f43e68d04ff77204b8
$ git push -f origin jdk11-macos

Then proceed to create a new pull request with NEW-BRANCH-NAME as the source branch and close this one.

@openjdk openjdk bot added the rfr label Dec 22, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 22, 2021

Webrevs

@lewurm
Copy link
Member

@lewurm lewurm commented Dec 28, 2021

I'm having troubles building it with Xcode 13.2:

=== Output from failing command(s) repeated here ===
* For target support_native_java.desktop_libsplashscreen_splashscreen_sys.o:
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:39: error: 'NSAlphaFirstBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-declarations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                      ^~~~~~~~~~~~~~~~~~~~~~~~
                                      NSBitmapFormatAlphaFirst
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:162:29: note: 'NSAlphaFirstBitmapFormat' has been expl
icitly marked deprecated here
static const NSBitmapFormat NSAlphaFirstBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaFirst", macos(10.5,10.14)) = NSBitmapFormatAlphaFirst;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:66: error: 'NSAlphaNonpremultipliedBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-decl
arations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                 NSBitmapFormatAlphaNonpremultiplied
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:163:29: note: 'NSAlphaNonpremultipliedBitmapFormat' ha
s been explicitly marked deprecated here
static const NSBitmapFormat NSAlphaNonpremultipliedBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaNonpremultiplied", macos(10.0,10.14)) = NSBitmapFormatAlphaNonpremultiplied;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:412:34: error: 'NSBorderlessWindowMask' is deprecated: first deprecated in macOS 10.12 [-Werror,-Wdeprecated-declarations]
   ... (rest of output omitted)

* All command lines available in /Users/lewurm/private/aarch64-port/build/macosx-aarch64-normal-server-release/make-support/failure-logs.
=== End of repeated output ===

Is this expected?

@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Dec 28, 2021

I'm having troubles building it with Xcode 13.2:

=== Output from failing command(s) repeated here ===
* For target support_native_java.desktop_libsplashscreen_splashscreen_sys.o:
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:39: error: 'NSAlphaFirstBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-declarations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                      ^~~~~~~~~~~~~~~~~~~~~~~~
                                      NSBitmapFormatAlphaFirst
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:162:29: note: 'NSAlphaFirstBitmapFormat' has been expl
icitly marked deprecated here
static const NSBitmapFormat NSAlphaFirstBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaFirst", macos(10.5,10.14)) = NSBitmapFormatAlphaFirst;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:274:66: error: 'NSAlphaNonpremultipliedBitmapFormat' is deprecated: first deprecated in macOS 10.14 [-Werror,-Wdeprecated-decl
arations]
                        bitmapFormat: NSAlphaFirstBitmapFormat | NSAlphaNonpremultipliedBitmapFormat
                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                 NSBitmapFormatAlphaNonpremultiplied
/Applications/Xcode13.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSBitmapImageRep.h:163:29: note: 'NSAlphaNonpremultipliedBitmapFormat' ha
s been explicitly marked deprecated here
static const NSBitmapFormat NSAlphaNonpremultipliedBitmapFormat API_DEPRECATED_WITH_REPLACEMENT("NSBitmapFormatAlphaNonpremultiplied", macos(10.0,10.14)) = NSBitmapFormatAlphaNonpremultiplied;
                            ^
/Users/lewurm/private/aarch64-port/src/java.desktop/macosx/native/libsplashscreen/splashscreen_sys.m:412:34: error: 'NSBorderlessWindowMask' is deprecated: first deprecated in macOS 10.12 [-Werror,-Wdeprecated-declarations]
   ... (rest of output omitted)

* All command lines available in /Users/lewurm/private/aarch64-port/build/macosx-aarch64-normal-server-release/make-support/failure-logs.
=== End of repeated output ===

Is this expected?

Yes, this is expected, it's same in upstream IIRC, need to wait for a patch there first.
Just use --disable-warnings-as-errors configure argument

@lewurm
Copy link
Member

@lewurm lewurm commented Jan 5, 2022

Running fastdebug tier1 on macOS/AArch64 looks good:

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
>> jtreg:test/hotspot/jtreg:tier1                     1239  1238     1     0 <<
>> jtreg:test/jdk:tier1                               1872  1866     6     0 <<
   jtreg:test/langtools:tier1                         3915  3915     0     0 
   jtreg:test/nashorn:tier1                              0     0     0     0
   jtreg:test/jaxp:tier1                                 0     0     0     0
==============================
TEST FAILURE

Those 6 failures in jdk:tier also happen on Linux/AArch64, so I'll ignore them. The hotspot failure happens on test/hotspot/jtreg/runtime/ErrorHandling/TestOnError.java with the following output:

TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: '^Test Succeeded' missing from stdout

Not sure what's going on.

Circeling back to the r18 issue mentioned here: openjdk/jdk11u-dev#301 (comment)
My understanding is that the current stable release of macOS (12.1 Monterey) should already be affected by this issue, as I haven't backported https://bugs.openjdk.java.net/browse/JDK-8274795 to jdk11 yet. Do you have some better reproducer other than running Intellij? Apparently none of the tier1 tests seem to trigger it.

Edit: The release build of this branch actually runs Intellij just fine.

@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Jan 5, 2022

Circeling back to the r18 issue mentioned here: openjdk/jdk11u-dev#301 (comment) My understanding is that the current stable release of macOS (12.1 Monterey) should already be affected by this issue, as I haven't backported https://bugs.openjdk.java.net/browse/JDK-8274795 to jdk11 yet. Do you have some better reproducer other than running Intellij? Apparently none of the tier1 tests seem to trigger it.

Edit: The release build of this branch actually runs Intellij just fine.

that issue was gone after final version of r18 patch were included into jdk11u
in fact the issue was with c2 registers allocations and some #ifdef APPLE not working as expected in ad files.

@ahmedmuhsin
Copy link

@ahmedmuhsin ahmedmuhsin commented Jan 5, 2022

Here are my tier2 tests results for the release build on Monterey 12.0.1/ Xcode 13.1:

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR   
   jtreg:test/hotspot/jtreg:tier2                      192   192     0     0   
>> jtreg:test/jdk:tier2                               3389  3383     4     2 <<
   jtreg:test/langtools:tier2                           12    12     0     0   
   jtreg:test/nashorn:tier2                             36    36     0     0   
   jtreg:test/jaxp:tier2                               438   438     0     0   
==============================
TEST FAILURE

Fail

  • java/net/NetworkInterface/UniqueMacAddressesTest.java: Tests that the MAC addresses returned by NetworkInterface.getNetworkInterfaces are unique for each adapter.
    • result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: mac address uniqueness test failed
  • java/net/Socket/LinkLocal.java: Connecting to a link-local IPv6 address should not causes a SocketException to be thrown.
    • result: Failed. Execution failed: `main' threw exception: java.net.SocketTimeoutException: Receive timed out
  • java/net/Socket/UrgentDataTest.java: TCP Urgent data support
    • result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Exception caught (UrgentDataTest.java:120)
  • java/nio/channels/Selector/OutOfBand.java: OOB data causes a SocketChannel, with OOBINLINE disabled, to be selected
    • result: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Received more than expected

Error

  • java/net/ipv6tests/UdpTest.java: IPv6 support for Windows XP and 2003 server
    • result: Error. Agent error: java.lang.Exception: Agent 26 timed out with a timeout of 480 seconds; check console log for any additional details
  • java/nio/channels/DatagramChannel/MinSendBufferSize.java: Check that new SO_SNDBUF limit on macOS is adhered to
    • result: Error. test was interrupted! (timeout)

@lewurm I am also getting the same TestOnError failure in hotspot:tier1 when I run the tests on the fastdebug build. However, I do not get the 6 langtools failures. All tier1 tests pass on the release build though.

@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Jan 6, 2022

Thanks, these tier2 failures for network tests are common for macos_intel and macos_aarch64, they might requres special setup of network interfaces on the host

TestOnError failure in hotspot:tier1 when I run the tests on the fastdebug build.

I'll check that, thanks.

@mlbridge
Copy link

@mlbridge mlbridge bot commented Jan 6, 2022

Mailing list message from Andrew Haley on aarch64-port-dev:

On 1/5/22 14:13, Bernhard Urban-Forster wrote:

My understanding is that the current stable release of macOS (12.1 Monterey) should already be affected by this issue, as I haven't backportedhttps://bugs.openjdk.java.net/browse/JDK-8274795 to jdk11 yet.

Is there an issue here that prevents 8274795 from being backported to 11? It's
obviously a bug. Are you waiting for the MacOS port?

--
Andrew Haley (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

adinn
adinn approved these changes Jan 6, 2022
Copy link
Collaborator

@adinn adinn left a comment

Approved

@openjdk
Copy link

@openjdk openjdk bot commented Jan 6, 2022

@VladimirKempik This change now passes all automated pre-integration checks.

After integration, the commit message for the final commit will be:

8253795: Implementation of JEP 391: macOS/AArch64 Port

Reviewed-by: adinn, burban

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the jdk11-macos branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the jdk11-macos branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Jan 6, 2022
@lewurm
Copy link
Member

@lewurm lewurm commented Jan 7, 2022

@lewurm I am also getting the same TestOnError failure in hotspot:tier1 when I run the tests on the fastdebug build. However, I do not get the 6 langtools failures. All tier1 tests pass on the release build though.

Thanks for looking at it @ahmedmuhsin, probably an issue with my setup then.

Edit: The release build of this branch actually runs Intellij just fine.

Running on that build for >48h now. Seems good 🙂 (it doesn't just sit in the background, but I'm actually using it).

Is there an issue here that prevents 8274795 from being backported to 11? It's
obviously a bug. Are you waiting for the MacOS port?

Yes, I was hoping the macOS port nicely crashes 😉 Here is the backport: openjdk/jdk11u-dev#744


@VladimirKempik not sure how you would like to proceed with this, do you want to merge this PR into jdk11-macos or do we just let it sit and bake here? Both is fine I think.

@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Jan 8, 2022

@lewurm just proceed with openjdk/jdk11u-dev#744 it's independent of this PR

@lewurm
Copy link
Member

@lewurm lewurm commented Jan 10, 2022

@lewurm just proceed with openjdk/jdk11u-dev#744 it's independent of this PR

I meant this PR: #14 🙂 I think there aren't any (strict) formal processes for the aarch64 project.

lewurm
lewurm approved these changes Jan 10, 2022
@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Jan 12, 2022

About TestOnError test failure,
this is a debug-only test and it really fails on macos-aarch64 fastdebug build.
Easy to reproduce command line:

./build/macosx-aarch64-normal-server-fastdebug/images/jdk/bin/java -XX:-TransmitErrorReport -XX:-CreateCoredumpOnCrash -XX:ErrorHandlerTest=12 -XX:OnError="echo Test" -version
results in
zsh: trace trap ./build/macosx-aarch64-normal-server-fastdebug/images/jdk/bin/java
and in logs one can see:
kernel AMFI: Denying core dump for pid 69372 (java)

This happens due to the miss of this backport : openjdk/jdk@c37eabe7

we have 3 options:

  1. backport that rather bigpatch into jdk11
  2. exclude this test on macos-aarch64
  3. change the test from -XX:ErrorHandlerTest=12 to -XX:ErrorHandlerTest=14 which makes it crash like expected.

@theRealAph @adinn and other, what do you think about this one ?

@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Jan 18, 2022

So what ErrorHandlerTest=12 does is basically this:
char * const dataPtr = NULL;
*dataPtr = '\0';

and this code alone ( in a simple app) does behave differently on mac_intel and mac_arm ( both with ulimit -c unlimited and user writable /cores folder)

mac_intel does this:
Segmentation fault: 11 (core dumped)
the core is then present in /cores folder

mac_arm does this:
zsh: segmentation fault ./testcrash
and printing this in dmesg:
[542308.736586]: AMFI: Denying core dump for pid 2283 (testcrash)testcrash[2283] Corpse allowed 1 of 5

so I believe the correct way is the way number 2
If no objections, I will update the PR in few days to ignore this test on mac_arm

@adinn
Copy link
Collaborator

@adinn adinn commented Jan 21, 2022

@VladimirKempik I agree the best approach is just to exclude this on macos-aarch64.

@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Jan 22, 2022

@theRealAph Can I push this now into aarch64-port repo?

@theRealAph
Copy link
Collaborator

@theRealAph theRealAph commented Jan 24, 2022

@theRealAph Can I push this now into aarch64-port repo?

Yes, thank you.

@VladimirKempik
Copy link
Author

@VladimirKempik VladimirKempik commented Jan 24, 2022

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Jan 24, 2022

Going to push as commit 54b8298.

@openjdk openjdk bot added the integrated label Jan 24, 2022
@openjdk openjdk bot closed this Jan 24, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Jan 24, 2022

@VladimirKempik Pushed as commit 54b8298.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
5 participants