AdoptOpenJDK + OpenJ9 11 to 15 java process hanging on mac os catalina

[vitorpavanelli]

Java -version output

openjdk version "14.0.2" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.2+12)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.21.0, JRE 14 Mac OS X amd64-64-Bit Compressed References 20200715_143 (JIT enabled, AOT enabled)
OpenJ9 - 34cf4c0
OMR - 113e54219
JCL - 1d231bd6a2 based on jdk-14.0.2+12)

Summary of problem

I am running a spring boot application and when doing control + C or stopping the application from within IntelliJ a java process keeps hanging and I need to kill it manually. I have tried version 11, 14 and 15 and the same happens.

I have attached screenshots for reference from the terminal.

If I run OpenJDK without OpenJ9 everything works fine.

Also, it could be something similar to this one #3468

Please, can someone take a look at it?

Thank you,
Vitor

Diagnostic files

[pshipton]

@tajila @babsingh fyi

[babsingh]

@vitorpavanelli Is the issue reproducible with any spring boot application? If not, can you provide your spring boot application for reproducing the issue?

[vitorpavanelli]

hi @babsingh! yes, you can go ahead and create a simple example from spring initialiser. You will notice when trying doing ctrl + c the java process won't quit. It seems whatever is waiting for the kill signal is not receiving or something got stuck. I even had this issue when running within IntelliJ IDE where I had to click twice on the stop button to actually kill the application.

[babsingh]

Thanks, @vitorpavanelli. We will look into this issue. fyi @dusanboskovic ^^^.

[dusanboskovic]

I was able to replicate this issue on Mac and zOS

[dusanboskovic]

I was able to replicate the issue on Linux and zOS using a test springboot application. Narrowed down the problem between these 2 IBM Java builds
SR6_FP11 - built June 1st 2020
SR6_FP15 - built July 24th 2020
On the open side, matched the dates from these builds with the nightly builds within the same time frame: https://adoptopenjdk.net/nightly.html?variant=openjdk8&jvmVariant=openj9
Narrowed it down to these builds:

Works as expected (Nightly build June 15):

openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-202006151723-b06)
Eclipse OpenJ9 VM (build master-cec5b1c53, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20200615_679 (JIT enabled, AOT enabled)
OpenJ9   - cec5b1c53
OMR      - 1db16ee21
JCL      - a6d49b5699 based on jdk8u262-b06)

Sometimes works (Nightly build June 16)

openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-202006161721-b06)
Eclipse OpenJ9 VM (build master-77c3af970, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20200616_680 (JIT enabled, AOT enabled)
OpenJ9   - 77c3af970
OMR      - fa22a8385
JCL      - a6d49b5699 based on jdk8u262-b06)

Never works (nightly build June 18)

openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-202006171720-b07)
Eclipse OpenJ9 VM (build master-50ba90c72, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20200617_681 (JIT enabled, AOT enabled)
OpenJ9   - 50ba90c72
OMR      - b884d8266
JCL      - 8d231b1446 based on jdk8u262-b07)

These are the git compare SHAs:
'https://github.com/eclipse/openj9/compare/cec5b1c53...50ba90c72'
'https://github.com/eclipse/openj9-omr/compare/1db16ee21...b884d8266' <-- probably nothing here
'https://github.com/ibmruntimes/openj9-openjdk-jdk8/compare/a6d49b5699...8d231b1446' <-- probably nothing here as well

Reverted @gacholio change: ac6141a
on the customer’s version that doesn’t work (SR6_FP15), and now the SIGTERM is properly handled
FYI: @babsingh @tajila @gacholio

Steps to reproduce

1. Very simple, you can use my springboot jar and run with any Java version above the IBM Java level SR6_FP15 or the Eclipse OpenJ9 release after the 17th of June.
2. Use the following command to execute the jar file: <path_to_java>/bin/java -jar spring-boot-0.0.1-SNAPSHOT.jar
3. On another terminal of the same machine, run ps -ef | grep spring and send a kill -15 <PID> to the springboot process. It should hang on anything above the releases mentioned above, but it should exit properly on anything before the releases above
   spring-boot-0.0.1-SNAPSHOT.jar.zip

[babsingh]

Summary: Reverting #9885 resolves the hang.

[gacholio]

That code has already been reverted. Current builds should work.

[dusanboskovic]

Ran the test scenario with a nightly build from the 20th of October and SIGTERM was properly handled. Seems like the fix will be officially released in one of the next versions of Eclipse OpenJ9

[babsingh]

Changes in #9885 were reverted in #10328 (4 Aug 2020).

[gacholio]

Two PRs fixed this - #10273 and #10328

[babsingh]

JDK14 is no longer supported since it only had a 6 month support life cycle. The latest release of JDK15 replaces JDK14 and was released on 23 Sept 20. It should have the above fixes.

The latest release of macOS x64 JDK 15 can be downloaded from Adopt: https://adoptopenjdk.net/archive.html?variant=openjdk15&jvmVariant=openj9.

@vitorpavanelli Can you please close this issue after verifying that the latest release of JDK15 fixes the hang?

[vitorpavanelli]

Hi @babsingh,

15 doesn't work. I actually installed it first and then tried older versions to make sure it wasn't an issue with 15. I understand 14 it isn't supported but if there will be no fixes for 11, 12, 13 and 14 then it should be removed from download page as well. At least 11 must be supported because it LTS.

Thank you for your hard work on this.

[gacholio]

@vitorpavanelli Can you please try the latest nightly build? The fix has not yet made it into a public release.

[babsingh]

Nightly builds: https://adoptopenjdk.net/nightly.html?variant=openjdk15&jvmVariant=openj9. Latest one was generated on 21 Oct 20.

At least 11 must be supported because it LTS.

LTS versions (8 & 11) will be supported along with JDK-latest. I didn't realize that the fix was not delivered to a release branch yet.

[babsingh]

it should be removed from download page

The old unsupported JDKs (9, 12, 13 and 14) are not publicized at Adopt's home page: https://adoptopenjdk.net/index.html?variant=openjdk8&jvmVariant=openj9. Only LTS (8 & 11) and JDK-latest versions are publicized.

I don't think the old unsupported JDKs will be removed. They will be retained for future reference. But, they won't be publicized at Adopt.

[vitorpavanelli]

it should be removed from download page

The old unsupported JDKs (9, 12, 13 and 14) are not publicized at Adopt's home page: https://adoptopenjdk.net/index.html?variant=openjdk8&jvmVariant=openj9. Only LTS (8 & 11) and JDK-latest versions are publicized.

I was able to access them in another manner, I don't recall where but anyway.

Your fix did work.

Thank you

[vitorpavanelli]

when can I download the release version, please?

[gacholio]

@pshipton When do you expect 0.23 to be available?

[pshipton]

I hope it will be available on Friday if nothing goes wrong, otherwise early next week.

[pshipton]

The 0.23 release occurred on Oct 23 with a jdk15 respin for Mac on Oct 26.

Release information available here https://github.com/eclipse/openj9/releases/tag/openj9-0.23.0