[Bug]: A fatal error has been detected by the Java Runtime Environment SIGSEGV (0xb)

[jebbench]

Please add the exact image (with tag) that you are using

eclipse-temurin:jdk-17

Please add the version of Docker you are running

Google Kubernetes Engine

What happened?

JVM crashed (crash log below) when under light load.

Unfortunately it was running on a container which was restarted after the crash so I can't access the error report files.

We've seen the crash occur twice, both times when the application was under light load.

The image was built using JIB (https://github.com/GoogleContainerTools/jib). We've now seen this occur in a container that was not built using JIB.

Heap usage was around 1gb of 2gb allocated.

Relevant log output

[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f3ea639d898]
#
# https://github.com/adoptium/adoptium-support/issues
# If you would like to submit a bug report, please visit:
#
# //replay_pid1.log
# Compiler replay data is saved as:
#
# //hs_err_pid1.log
# An error report file with more information is saved as:
#
# JFR recording file will be written. Location: //hs_err_pid1.jfr
#
# Core dump will be written. Default location: /core.%e.1.%t
#
# C 0x0000000000000000
# Problematic frame:
# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (17.0.5+8, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# JRE version: OpenJDK Runtime Environment Temurin-17.0.5+8 (17.0.5+8) (build 17.0.5+8)
#
# SIGSEGV (0xb) at pc=0x0000000000000000, pid=1, tid=19
#
# A fatal error has been detected by the Java Runtime Environment:
#

[karianna]

@jebbench We'll need the crash log to investigate. On the off chance this is fixed in 17.0.6 then you can wait a couple of weeks for that release.

[jebbench]

@karianna this is still occuring using 17.0.6.

I will see if there's a way to retrieve the crash logs.

[jebbench]

@karianna I've been able to get the crash log:
hs_err_pid1.log

Let me know if there's anything else we can do or provide to help.

[karianna]

Ah fun this is a Kotlin sourced crash, what version of Kotlin are you running?

[jebbench]

Afraid so, the application was built using Kotlin 1.7.21.

[karianna]

Afraid so, the application was built using Kotlin 1.7.21.

Any chance you're able to move to 1.8.10?

[jebbench]

Yes - we should be able to migrate to 1.8.10, but it could be a few weeks before I can get another crash log unless we get lucky and see one in our test environment (unless the upgrade fixes it 🤞).

I'll start migrating our projects.

[jebbench]

Crash report using Kotlin 1.8.10.
hs_err_pid1.log

[karianna]

Interesting, also was reported #659

[karianna]

@jebbench Apologies for the run around but https://youtrack.jetbrains.com/issues/kt is the place you'll want to report this as Kotlin is emitting something that the C2 compiler doesn't like.

[jebbench]

Thanks @karianna, I have raised this as https://youtrack.jetbrains.com/issue/KT-56975/JVM-A-fatal-error-has-been-detected-by-the-Java-Runtime-Environment-SIGSEGV-0xb

[karianna]

Thanks, closing here (but if the Kotlin folks bounce it back we can look at opening up at OpenJDK proper itself).

[ilmirus]

@karianna

Hi, Kotlin Team here. Yes, it looks very much as a OpenJDK bug. The thing with coroutines is that we generate code, which is not possible to write in Java - we have non-canonical loops or jumps to catch blocks, which are likely to not be covered by tests. There was even an anekdot, that Graal could not compile coroutines code and run the whole module in interpreter mode. Additionally, if there was a bug in generated bytecode, it would lead to VerifyError. Also, there is an issue reported with same JDK version and same symptoms - SIGSEGV in C2 on coroutines code and is very sporadic https://youtrack.jetbrains.com/issue/KT-54693/SIGSEGV-0xb-at-pc0x0000000000000000-C2-CompilerThread0.

[karianna]

Thanks @ilmirus - I'll get this reported upstream!

[karianna]

https://bugs.openjdk.org/browse/JDK-8303279

[jebbench]

This is still an issue in OpenJDK 64-Bit Server VM Temurin-17.0.7+7 (but has apparently been resolved in OpenJDK 19 - I've not been able to verify though).

[karianna]

@jebbench Would be good to understand the fix in 19 - is there a bug report or mail thread to reference?

[jebbench]

@karianna the thread I had been reading was this one: grpc/grpc-kotlin#370 (comment)

[karianna]

https://bugs.openjdk.org/browse/JDK-8303279

[simonis]

Fixed by openjdk/jdk#14600 in JDK 22.

Still needs to be downported to 17 & 21.

[gdams]

Fixed by openjdk/jdk#14600 in JDK 22.

Still needs to be downported to 17 & 21.

the 17 backport PR is here: openjdk/jdk17u-dev#1580

[simonis]

Notice that this doesn't doesn't affect 11 because it was introduced by https://bugs.openjdk.org/browse/JDK-8238691 which only went int 15.

[gdams]

Notice that this doesn't doesn't affect 11 because it was introduced by https://bugs.openjdk.org/browse/JDK-8238691 which only went int 15.

I was just thinking the same thing, I'll remove the label

[karianna]

Do we close or does it need a 21 backport?

[gdams]

It went into 21 here: openjdk/jdk21#119