8369505: jhsdb jstack cannot handle continuation stub #27728

YaSuenag · 2025-10-09T14:11:39Z

I tried to get mixed thread dump of the application which runs virtual threads (see Test.java on JBS) via jhsdb jstack --mixed, then I got following message:

sun.jvm.hotspot.utilities.AssertionFailure: must have non-zero frame size
        at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.senderForCompiledFrame(X86Frame.java:374)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.x86.X86Frame.sender(X86Frame.java:273)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.sender(Frame.java:225)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:230)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:120)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:150)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.initJFrameCache(PStack.java:224)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:73)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:65)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:60)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:306)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:507)

And also I got following (strange) stacks which causes AssersionFailure in above:

----------------- 70094 -----------------
"ForkJoinPool-1-worker-4" #32 daemon prio=5 tid=0x00007f8f5c371660 nid=70094 runnable [0x00007f8f406d9000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x00007f8f64658462 __syscall_cancel_arch + 0x32
0x00007f8f6464c75c __internal_syscall_cancel + 0x5c
0x00007f8f646a8c37 __GI___nanosleep + 0x17
0x00007f8f646bb14e __sleep + 0x3e
0x00007f8f4b3a8e1e <nep_invoker_blob>
0x00007f8f4b33fe48 * java.lang.invoke.LambdaForm$MH+0x000000000c047000.invoke(java.lang.Object, long, int) bci:10 (Interpreted frame)
0x00007f8f4b33fe48 * java.lang.invoke.LambdaForm$MH+0x000000000c050800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Interpreted frame)
0x00007f8f4b33fe48 * jdk.internal.foreign.abi.DowncallStub+0x000000000c048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x00007f8f4b33fe48 * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Interpreted frame)
0x00007f8f4b33fe48 * java.lang.invoke.LambdaForm$MH+0x000000000c04e400.invoke(java.lang.Object, int) bci:44 (Interpreted frame)
0x00007f8f4b33fd00 * java.lang.invoke.LambdaForm$MH+0x000000000c04c400.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Interpreted frame)
0x00007f8f4b33fd00 * Test.run() bci:21 line:28 (Interpreted frame)
0x00007f8f4b33e098 <StubRoutines (continuation stubs)>
0x00007f8f4b33fd00 <interpreter> return entry points
0x00007f8f4b33fd00 <interpreter> return entry points
0x00007f8f4b33fd00 <interpreter> return entry points
0x00007f8f4b340206 <interpreter> return entry points
0x00007f8f4b33fe9a <interpreter> return entry points
0x00007f8f4b33fe9a <interpreter> return entry points
0x00007f8f4b33fe48 <interpreter> return entry points
0x00007f8f4b33fd00 <interpreter> return entry points
0x00007f8f4b33fd00 <interpreter> return entry points
0x00007f8f4b33fd00 <interpreter> return entry points
0x00007f8f4b3386fd <StubRoutines (initial stubs)>
0x00007f8f62bc0a7e JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x4ce
0x00007f8f62bc11b3 JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x2d3
0x00007f8f62bc17bb JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0xab
0x00007f8f62d83a90 thread_entry(JavaThread*, JavaThread*) + 0xd0
0x00007f8f62c02806 JavaThread::thread_main_inner() + 0x256
0x00007f8f63865b57 Thread::call_run() + 0xb7
0x00007f8f632fc588 thread_native_entry(Thread*) + 0x128
0x00007f8f6464ff54 start_thread + 0x2e4

According to stack layout described in continuationFreezeThaw.cpp and stub implementation in StubGenerator::generate_cont_thaw() in stubGenerator_x86_64.cpp, we cannot calculate caller SP from CodeBlob for continuation stub. We need to restore SP from _cont_entry in JavaThread.

After this fix, we can see following stacks:

----------------- 39371 -----------------
"ForkJoinPool-1-worker-1" #27 daemon prio=5 tid=0x00007fe83036f230 nid=39371 runnable [0x00007fe815e06000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x00007fe839b17462      __syscall_cancel_arch + 0x32
0x00007fe839b0b75c      __internal_syscall_cancel + 0x5c
0x00007fe839b67c37      __GI___nanosleep + 0x17
0x00007fe839b7a14e      __sleep + 0x3e
0x00007fe81f3a859e      <nep_invoker_blob>
0x00007fe81f33fe48      * java.lang.invoke.LambdaForm$MH+0x0000000041047000.invoke(java.lang.Object, long, int) bci:10 (Interpreted frame)
0x00007fe81f33fe48      * java.lang.invoke.LambdaForm$MH+0x0000000041051400.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Interpreted frame)
0x00007fe81f33fe48      * jdk.internal.foreign.abi.DowncallStub+0x0000000041048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x00007fe81f33fe48      * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Interpreted frame)
0x00007fe81f33fe48      * java.lang.invoke.LambdaForm$MH+0x000000004104e800.invoke(java.lang.Object, int) bci:44 (Interpreted frame)
0x00007fe81f33fd00      * java.lang.invoke.LambdaForm$MH+0x000000004104d800.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Interpreted frame)
0x00007fe81f33fd00      * Test.run() bci:21 line:28 (Interpreted frame)
0x00007fe81f33e098      <StubRoutines (continuation stubs)>
0x00007fe81f33fd00      * jdk.internal.vm.Continuation.run() bci:122 line:248 (Interpreted frame)
0x00007fe81f33fd00      * java.lang.VirtualThread.runContinuation() bci:100 line:293 (Interpreted frame)
0x00007fe81f33fd00      * java.lang.VirtualThread$$Lambda+0x0000000041029b58.run() bci:4 (Interpreted frame)
0x00007fe81f340206      * java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() bci:4 line:1753 (Interpreted frame)
0x00007fe81f33fe9a      * java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() bci:1 line:1745 (Interpreted frame)
0x00007fe81f33fe9a      * java.util.concurrent.ForkJoinTask$InterruptibleTask.exec() bci:51 line:1662 (Interpreted frame)
0x00007fe81f33fe48      * java.util.concurrent.ForkJoinTask.doExec() bci:10 line:511 (Interpreted frame)
0x00007fe81f33fd00      * java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) bci:5 line:1450 (Interpreted frame)
0x00007fe81f33fd00      * java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) bci:364 line:2019 (Interpreted frame)
0x00007fe81f33fd00      * java.util.concurrent.ForkJoinWorkerThread.run() bci:31 line:187 (Interpreted frame)
0x00007fe81f3386fd      <StubRoutines (initial stubs)>
0x00007fe837fc0a7e      JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x4ce
0x00007fe837fc11b3      JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x2d3
0x00007fe837fc17bb      JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0xab
0x00007fe838183a90      thread_entry(JavaThread*, JavaThread*) + 0xd0
0x00007fe838002806      JavaThread::thread_main_inner() + 0x256
0x00007fe838c65b57      Thread::call_run() + 0xb7
0x00007fe8386fc588      thread_native_entry(Thread*) + 0x128
0x00007fe839b0ef54      start_thread + 0x2e4

I saw this issue on Linux AMD64. I'm not sure but it might happen on another platforms.

I haven't yet created a complete reproducer, but Test.java on JBS can be reproduced with high probability.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Warning

⚠️ Found leading lowercase letter in issue title for 8369505: jhsdb jstack cannot handle continuation stub

Issue

JDK-8369505: jhsdb jstack cannot handle continuation stub (Bug - P4)

Reviewers

Chris Plummer (@plummercj - Reviewer)
Patricio Chilano Mateo (@pchilano - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/27728/head:pull/27728
$ git checkout pull/27728

Update a local copy of the PR:
$ git checkout pull/27728
$ git pull https://git.openjdk.org/jdk.git pull/27728/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 27728

View PR using the GUI difftool:
$ git pr show -t 27728

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/27728.diff

Using Webrev

Link to Webrev Comment

…x AMD64

bridgekeeper · 2025-10-09T14:13:31Z

👋 Welcome back ysuenaga! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-10-09T14:13:34Z

@YaSuenag This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8369505: jhsdb jstack cannot handle continuation stub

Reviewed-by: cjplummer, pchilanomate

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 112 new commits pushed to the master branch:

c9cbd31: 8307495: Specialize atomic bitset functions for aix-ppc
bfe6937: 8369444: JavaFrameAnchor on PPC64 has unnecessary barriers
1bd814c: 8364660: ClassVerifier::ends_in_athrow() should be removed
... and 109 more: https://git.openjdk.org/jdk/compare/c12f0e74a9acfaa91b537853143ec94f143fe316...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2025-10-09T14:14:33Z

@YaSuenag The following labels will be automatically applied to this pull request:

hotspot-runtime
serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

mlbridge · 2025-10-09T23:34:19Z

Webrevs

pchilano · 2025-10-10T15:19:54Z

I can reproduce it with your attached test too. The issue happens when the return pc of the sender frame is the return barrier, which will be the case if the virtual thread was unmounted at some earlier point and there are still frames freezed in the stackChunk (see frame::sender_for_interpreter_frame and frame::sender_for_compiled_frame for handling this case in the VM). In your test, unmounting can happen at the System.out.println call.
So this issue will be present in the other platforms too. I tested the fix for x64 and looks good.

plummercj · 2025-10-10T19:22:35Z

I'm seeing a similar exception on macosx-aarch64 with your test, however, I don't see all the frames that you are seeing. The stack dump stops at Test.run().

Also, on threads that are fully dumped, the output is different than you working dump above. I'm not sure if this is expected:

"ForkJoinPool-1-worker-6" #33 daemon prio=5 tid=0x0000000147856810 nid=41731 runnable [0x0000000172df5000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
 - java.lang.invoke.LambdaForm$MH+0x0000000801047000.invoke(java.lang.Object, long, int) @bci=10 (Interpreted frame)
 - java.lang.invoke.LambdaForm$MH+0x0000000801054800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) @bci=21 (Interpreted frame)
 - jdk.internal.foreign.abi.DowncallStub+0x0000000801048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) @bci=44 (Interpreted frame)
 - java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) @bci=14 (Interpreted frame)
 - java.lang.invoke.LambdaForm$MH+0x000000080104f400.invoke(java.lang.Object, int) @bci=44 (Interpreted frame)
 - java.lang.invoke.LambdaForm$MH+0x000000080104e000.invoke_MT(java.lang.Object, int, java.lang.Object) @bci=18 (Interpreted frame)
 - Test.run() @bci=21, line=28 (Interpreted frame)
 - java.lang.Thread.runWith(java.lang.Object, java.lang.Runnable) @bci=5, line=1487 (Interpreted frame)
 - java.lang.VirtualThread.run(java.lang.Runnable) @bci=62, line=456 (Interpreted frame)
 - java.lang.VirtualThread$VThreadContinuation$1.run() @bci=15, line=248 (Interpreted frame)
 - jdk.internal.vm.Continuation.enter0() @bci=4, line=322 (Interpreted frame)
 - jdk.internal.vm.Continuation.enter(jdk.internal.vm.Continuation, boolean) @bci=1, line=313 (Interpreted frame)
 - jdk.internal.vm.Continuation.enterSpecial(jdk.internal.vm.Continuation, boolean, boolean) @bci=0 (Compiled frame)
 - jdk.internal.vm.Continuation.run() @bci=122, line=248 (Interpreted frame)
 - java.lang.VirtualThread.runContinuation() @bci=100, line=293 (Interpreted frame)
 - java.lang.VirtualThread$$Lambda+0x0000000801027670.run() @bci=4 (Interpreted frame)
 - java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() @bci=4, line=1753 (Interpreted frame)
 - java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() @bci=1, line=1745 (Interpreted frame)
 - java.util.concurrent.ForkJoinTask$InterruptibleTask.exec() @bci=51, line=1662 (Interpreted frame)
 - java.util.concurrent.ForkJoinTask.doExec() @bci=10, line=511 (Interpreted frame)
 - java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) @bci=5, line=1450 (Interpreted frame)
 - java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) @bci=364, line=2019 (Interpreted frame)
 - java.util.concurrent.ForkJoinWorkerThread.run() @bci=31, line=187 (Interpreted frame)

plummercj · 2025-10-10T19:24:07Z

I can see if your changes are applicable to aarch64. If they work, then great, if not, I'll probably need some help getting it right.

plummercj · 2025-10-10T19:27:57Z

Can you convert your test to JTREG? It would make it a lot easier to test fixes on all platforms. Also, the fix is a bit fragile and it would be nice to quickly detect if it breaks again in the future.

plummercj · 2025-10-10T19:52:52Z

I applied the changes to AARCH64Frame.java and they seem to fix the issue with the exception. I'm seeing two different stack traces (see below), and neither is identical to your fixed stack trace. I don't see <StubRoutines (continuation stubs)>. Instead I see a frame for jdk.internal.vm.Continuation.enterSpecial(). Also, I don't see <StubRoutines (initial stubs)> or any of the frames that come after it. Maybe these are just expected platform differences.

"ForkJoinPool-1-worker-5" #34 daemon prio=5 tid=0x0000000159051c10 nid=34819 runnable [0x0000000172761000]
java.lang.Thread.State: RUNNABLE
JavaThread state: _thread_in_native

java.lang.invoke.LambdaForm$MH+0x000003fe01047000.invoke(java.lang.Object, long, int) @bci=10 (Interpreted frame)
java.lang.invoke.LambdaForm$MH+0x000003fe01054800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) @bci=21 (Interpreted frame)
jdk.internal.foreign.abi.DowncallStub+0x000003fe01048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) @bci=44 (Interpreted frame)
java.lang.invoke.LambdaForm$DMH+0x000003fe01048400.invokeStaticInit(java.lang.Object, java.lang.Object, java.lang.Object, int) @bci=14 (Interpreted frame)
java.lang.invoke.LambdaForm$MH+0x000003fe0104f400.invoke(java.lang.Object, int) @bci=44 (Interpreted frame)
java.lang.invoke.LambdaForm$MH+0x000003fe0104e400.invoke_MT(java.lang.Object, int, java.lang.Object) @bci=18 (Interpreted frame)
Test.run() @bci=21, line=28 (Interpreted frame)
jdk.internal.vm.Continuation.enterSpecial(jdk.internal.vm.Continuation, boolean, boolean) @bci=0 (Compiled frame)
jdk.internal.vm.Continuation.run() @bci=152, line=251 (Interpreted frame)
java.lang.VirtualThread.runContinuation() @bci=100, line=293 (Interpreted frame)
java.lang.VirtualThread$$Lambda+0x000003fe01027670.run() @bci=4 (Interpreted frame)
java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() @bci=4, line=1753 (Interpreted frame)
java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() @bci=1, line=1745 (Interpreted frame)
java.util.concurrent.ForkJoinTask$InterruptibleTask.exec() @bci=51, line=1662 (Interpreted frame)
java.util.concurrent.ForkJoinTask.doExec() @bci=10, line=511 (Interpreted frame)
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) @bci=5, line=1450 (Interpreted frame)
java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) @bci=364, line=2019 (Interpreted frame)
java.util.concurrent.ForkJoinWorkerThread.run() @bci=31, line=187 (Interpreted frame)

"ForkJoinPool-1-worker-6" #36 daemon prio=5 tid=0x0000000159052610 nid=42243 runnable [0x000000017296d000]
java.lang.Thread.State: RUNNABLE
JavaThread state: _thread_in_native

java.lang.invoke.LambdaForm$MH+0x000003fe01047000.invoke(java.lang.Object, long, int) @bci=10 (Interpreted frame)
java.lang.invoke.LambdaForm$MH+0x000003fe01054800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) @bci=21 (Interpreted frame)
jdk.internal.foreign.abi.DowncallStub+0x000003fe01048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) @bci=44 (Interpreted frame)
java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) @bci=14 (Interpreted frame)
java.lang.invoke.LambdaForm$MH+0x000003fe01050400.invoke(java.lang.Object, int) @bci=44 (Interpreted frame)
java.lang.invoke.LambdaForm$MH+0x000003fe0104e400.invoke_MT(java.lang.Object, int, java.lang.Object) @bci=18 (Interpreted frame)
Test.run() @bci=21, line=28 (Interpreted frame)
java.lang.Thread.runWith(java.lang.Object, java.lang.Runnable) @bci=5, line=1487 (Interpreted frame)
java.lang.VirtualThread.run(java.lang.Runnable) @bci=62, line=456 (Interpreted frame)
java.lang.VirtualThread$VThreadContinuation$1.run() @bci=15, line=248 (Interpreted frame)
jdk.internal.vm.Continuation.enter0() @bci=4, line=322 (Interpreted frame)
jdk.internal.vm.Continuation.enter(jdk.internal.vm.Continuation, boolean) @bci=1, line=313 (Interpreted frame)
jdk.internal.vm.Continuation.enterSpecial(jdk.internal.vm.Continuation, boolean, boolean) @bci=0 (Compiled frame)
jdk.internal.vm.Continuation.run() @bci=122, line=248 (Interpreted frame)
java.lang.VirtualThread.runContinuation() @bci=100, line=293 (Interpreted frame)
java.lang.VirtualThread$$Lambda+0x000003fe01027670.run() @bci=4 (Interpreted frame)
java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() @bci=4, line=1753 (Interpreted frame)
java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() @bci=1, line=1745 (Interpreted frame)
java.util.concurrent.ForkJoinTask$InterruptibleTask.exec() @bci=51, line=1662 (Interpreted frame)
java.util.concurrent.ForkJoinTask.doExec() @bci=10, line=511 (Interpreted frame)
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) @bci=5, line=1450 (Interpreted frame)
java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) @bci=364, line=2019 (Interpreted frame)
java.util.concurrent.ForkJoinWorkerThread.run() @bci=31, line=187 (Interpreted frame)

pchilano · 2025-10-10T20:38:51Z

I applied the changes to AARCH64Frame.java and they seem to fix the issue with the exception. I'm seeing two different stack traces (see below), and neither is identical to your fixed stack trace. I don't see <StubRoutines (continuation stubs)>. Instead I see a frame for jdk.internal.vm.Continuation.enterSpecial(). Also, I don't see <StubRoutines (initial stubs)> or any of the frames that come after it. Maybe these are just expected platform differences.

These two different stack traces are expected and depend on whether the vthread was unmounted or not. With the current test this is timing dependent. You can run the test commenting out the System.out.println call, and then again replacing it with Thread.yield(). The string <StubRoutines (continuation stubs)> does look like from a previous version though. Even on x64 I see the enterSpecial frame printed out, and it matches what I would expect based on the patch.

YaSuenag · 2025-10-11T02:42:15Z

@plummercj Did you run jhsdb jstack --mixed ? StubRoutine would appear in mixed mode only I think (in Linux, at least).

Thanks @pchilano for explaining the problem, I understood it relates to unmount, it is triggered by System.out.println and we can replace it to Thread.yield. I can convert Test.java to JTREG test of course, However I'm not sure how we can reproduce this issue 100%. I guess the problem is very similar with race-condition - dependent on timing. Do you have any idea to happen the problem exactly?

YaSuenag · 2025-10-11T03:13:39Z

I saw same issue in JDK 25 on Fedora Rawhide on Raspberry Pi 4.
I will try to fix on AArch64.

Server compiler detected.
JVM version is 25+36-3489
sun.jvm.hotspot.utilities.AssertionFailure: must have non-zero frame size
        at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.aarch64.AARCH64Frame.senderForCompiledFrame(AARCH64Frame.java:414)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.aarch64.AARCH64Frame.sender(AARCH64Frame.java:295)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.sender(Frame.java:207)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:212)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:120)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:150)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.initJFrameCache(PStack.java:224)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:73)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:65)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:60)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:306)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:507)

----------------- 1884 -----------------
"ForkJoinPool-1-worker-4" #30 daemon prio=5 tid=0x0000ffff982c0ce0 nid=1884 runnable [0x0000ffff4d20b000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x0000ffffa03282e8      __syscall_cancel_arch + 0x28
0x0000ffffa0358b7c      __clock_nanosleep + 0x3c
0x0000ffffa0363740      __GI___nanosleep + 0x20
0x0000ffffa0373f50      __sleep + 0x50
0x0000ffff87c4c878      <nep_invoker_blob>
0x0000ffff87b3b910      * java.lang.invoke.LambdaForm$MH+0x0000004001047000.invoke(java.lang.Object, long, int) bci:10 (Interpreted frame)
0x0000ffff87b3b910      * java.lang.invoke.LambdaForm$MH+0x000000400104fc00.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Interpreted frame)
0x0000ffff87b3b910      * jdk.internal.foreign.abi.DowncallStub+0x0000004001048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x0000ffff87b3b910      * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Interpreted frame)
0x0000ffff87b3b910      * java.lang.invoke.LambdaForm$MH+0x000000400104dc00.invoke(java.lang.Object, int) bci:44 (Interpreted frame)
0x0000ffff87b3ba90      * java.lang.invoke.LambdaForm$MH+0x000000400104c400.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Interpreted frame)
0x0000ffff87b3ba90      * Test.run() bci:21 line:28 (Interpreted frame)
0x0000ffff87b3a080      <StubRoutines (continuationstubs)>
0x0000ffff87b3ba90      <interpreter> return entry points
0x0000ffff87b3ba90      <interpreter> return entry points
0x0000ffff87b3ba90      <interpreter> return entry points
0x0000ffff87b3c030      <interpreter> return entry points
0x0000ffff87b3b820      <interpreter> return entry points
0x0000ffff87b3b820      <interpreter> return entry points
0x0000ffff87b3b910      <interpreter> return entry points
0x0000ffff87b3ba90      <interpreter> return entry points
0x0000ffff87b3ba90      <interpreter> return entry points
0x0000ffff87b3ba90      <interpreter> return entry points
0x0000ffff87b37154      <StubRoutines (initialstubs)>
0x0000ffff9f620198      JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x248
0x0000ffff9f621824      JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0x174
0x0000ffff9f6fa7f8      thread_entry(JavaThread*, JavaThread*) + 0x8c
0x0000ffff9f639d88      JavaThread::thread_main_inner() [clone .part.0] + 0xa4
0x0000ffff9fbbc308      Thread::call_run() + 0xa8
0x0000ffff9fa006a8      thread_native_entry(Thread*) + 0xd8
0x0000ffffa031e8d4      start_thread + 0x404

plummercj · 2025-10-11T05:09:31Z

@plummercj Did you run jhsdb jstack --mixed ? StubRoutine would appear in mixed mode only I think (in Linux, at least).

I did not, and unfortunately --mixed is not support on OSX.

YaSuenag · 2025-10-11T13:51:59Z

I fixed for AArch64, and it works fine (in below) on Raspberry Pi 4.

----------------- 1687 -----------------
"ForkJoinPool-1-worker-2" #28 daemon prio=5 tid=0x0000ffff9c697a10 nid=1687 runnable [0x0000ffff70def000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x0000ffffa3b382e8      __syscall_cancel_arch + 0x28
0x0000ffffa3b68b7c      __clock_nanosleep + 0x3c
0x0000ffffa3b73740      __GI___nanosleep + 0x20
0x0000ffffa3b83f50      __sleep + 0x50
0x0000ffff8b15b278      <nep_invoker_blob>
0x0000ffff8b13bd54      * java.lang.invoke.LambdaForm$MH+0x0000000010087000.invoke(java.lang.Object, long, int) bci:10 (Interpreted frame)
0x0000ffff8b13bd54      * java.lang.invoke.LambdaForm$MH+0x0000000010090000.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Interpreted frame)
0x0000ffff8b13bd54      * jdk.internal.foreign.abi.DowncallStub+0x0000000010088000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x0000ffff8b13bd54      * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Interpreted frame)
0x0000ffff8b13bd54      * java.lang.invoke.LambdaForm$MH+0x000000001008e400.invoke(java.lang.Object, int) bci:44 (Interpreted frame)
0x0000ffff8b13bf24      * java.lang.invoke.LambdaForm$MH+0x000000001008cc00.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Interpreted frame)
0x0000ffff8b13bf24      * Test.run() bci:21 line:28 (Interpreted frame)
0x0000ffff8b13a3b8      <StubRoutines (continuation stubs)>
0x0000ffff8b13bf24      * jdk.internal.vm.Continuation.run() bci:122 line:248 (Interpreted frame)
0x0000ffff8b13bf24      * java.lang.VirtualThread.runContinuation() bci:100 line:293 (Interpreted frame)
0x0000ffff8b13bf24      * java.lang.VirtualThread$$Lambda+0x00000000100fba70.run() bci:4 (Interpreted frame)
0x0000ffff8b13c5b4      * java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() bci:4 line:1753 (Interpreted frame)
0x0000ffff8b13bc50      * java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute() bci:1 line:1745 (Interpreted frame)
0x0000ffff8b13bc50      * java.util.concurrent.ForkJoinTask$InterruptibleTask.exec() bci:51 line:1662 (Interpreted frame)
0x0000ffff8b13bd54      * java.util.concurrent.ForkJoinTask.doExec() bci:10 line:511 (Interpreted frame)
0x0000ffff8b13bf24      * java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) bci:5 line:1450 (Interpreted frame)
0x0000ffff8b13bf24      * java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) bci:364 line:2019 (Interpreted frame)
0x0000ffff8b13bf24      * java.util.concurrent.ForkJoinWorkerThread.run() bci:31 line:187 (Interpreted frame)
0x0000ffff8b13749c      <StubRoutines (initial stubs)>
0x0000ffffa24d15c4      JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x474
0x0000ffffa24d1c48      JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x278
0x0000ffffa24d21e4      JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0x90
0x0000ffffa2668a14      thread_entry(JavaThread*, JavaThread*) + 0xc4
0x0000ffffa2510d18      JavaThread::thread_main_inner() + 0x104
0x0000ffffa30399fc      Thread::call_run() + 0xac
0x0000ffffa2b6c69c      thread_native_entry(Thread*) + 0x12c
0x0000ffffa3b2e8d4      start_thread + 0x404

YaSuenag · 2025-10-11T14:54:05Z

@pchilano I reproduce the problem with Thread.yield(). Thanks! So I added test in the latest commit. I believe it works on both AMD64 and AArch64.

RealFYang · 2025-10-13T02:09:50Z

@YaSuenag :
Hi, I tried on linux-riscv64 and I see the assertion failure is triggering on this platform as well.
Seems that your changes are applicable to riscv64 with minor tweak. Maybe you can add this? Thanks.

diff --git a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/riscv64/RISCV64Frame.java b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/riscv64/RISCV64Frame.java
index e02e056f028..44c8f4c679c 100644
--- a/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/riscv64/RISCV64Frame.java
+++ b/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/riscv64/RISCV64Frame.java
@@ -262,7 +262,13 @@ public Frame sender(RegisterMap regMap, CodeBlob cb) {
     }

     if (cb != null) {
-      return cb.isUpcallStub() ? senderForUpcallStub(map, (UpcallStub)cb) : senderForCompiledFrame(map, cb);
+      if (cb.isUpcallStub()) {
+        return senderForUpcallStub(map, (UpcallStub)cb);
+      } else if (cb.isContinuationStub()) {
+        return senderForContinuationStub(map, cb);
+      } else {
+        return senderForCompiledFrame(map, cb);
+      }
     }

     // Must be native-compiled frame, i.e. the marshaling code for native
@@ -348,6 +354,16 @@ private void updateMapWithSavedLink(RegisterMap map, Address savedFPAddr) {
     map.setLocation(fp, savedFPAddr);
   }

+  private Frame senderForContinuationStub(RISCV64RegisterMap map, CodeBlob cb) {
+    var contEntry = map.getThread().getContEntry();
+
+    Address senderSP = contEntry.getEntrySP();
+    Address senderPC = contEntry.getEntryPC();
+    Address senderFP = contEntry.getEntryFP();
+
+    return new RISCV64Frame(senderSP, senderFP, senderPC);
+  }
+
   private Frame senderForCompiledFrame(RISCV64RegisterMap map, CodeBlob cb) {
     if (DEBUG) {
       System.out.println("senderForCompiledFrame");
diff --git a/test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackMixedWithVirtualThread.java b/test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackMixedWithVirtualThread.java
index 2565fdf9056..9b4fa067dc0 100644
--- a/test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackMixedWithVirtualThread.java
+++ b/test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackMixedWithVirtualThread.java
@@ -37,7 +37,7 @@
  * @test
  * @bug 8369505
  * @requires (os.family == "linux") & (vm.hasSA)
- * @requires (os.arch == "amd64" | os.arch == "aarch64")
+ * @requires (os.arch == "amd64" | os.arch == "aarch64" | os.arch == "riscv64")
  * @library /test/lib
  * @run driver TestJhsdbJstackMixedWithVirtualThread
  */

YaSuenag · 2025-10-13T03:04:21Z

@RealFYang Thanks a lot! I applied your change to this PR!

test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackMixedWithVirtualThread.java

test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackWithVirtualThread.java

test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackMixedWithVirtualThread.java

YaSuenag · 2025-10-14T01:01:15Z

@plummercj Thanks for your comment! I revised testcase in the latest commit. Could you review again?

pchilano · 2025-10-14T18:12:09Z

@plummercj Did you run jhsdb jstack --mixed ? StubRoutine would appear in mixed mode only I think (in Linux, at least).

I did not, and unfortunately --mixed is not support on OSX.

Ok, that’s why I wasn’t seeing <StubRoutines (continuation stubs)> either. I removed —mixed because it wasn’t working for me, only the top native frame was shown in the stacktrace (on linux-x64). I could still reproduce the assert without —mixed so I continued testing that way assuming the output for the enterSpecial frame would be the same in both modes.
Now, testing on linux-aarch64, —mixed mode works and I can see <StubRoutines (continuation stubs)> for the enterSpecial frame. Looking at the code, I see the difference with mixed mode is the way we walk the stack. Even for Java frames the sender is always obtained through this method [1], relying basically on the FP. Then once we get to the frame with pc equal to the return barrier, here [2] we will just branch to the else case and print it as a continuation stub instead of the enterSpecial frame.
I was also curious how we can walk compiled frames this way, since the FP will not contain in general a valid link unless -XX:+PreserveFramePointer is set. So I added -Xcomp to LingeredApp.startApp and I see that we indeed fail to walk the stack:

----------------- 2943661 -----------------
"ForkJoinPool-1-worker-2" #30 daemon prio=5 tid=0x0000ffff08007f30 nid=2943661 runnable [0x0000ffff695e1000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x0000ffff9bb40b24	__clock_nanosleep + 0x84
0x0000ffff9bb45c0c	__GI___nanosleep + 0x1c
0x0000ffff9bb45acc	__sleep + 0x4c
0x0000ffff836fb338	<nep_invoker_blob>
0x0000ffff7c802ef8	* java.lang.invoke.LambdaForm$MH+0x0000000601047800.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
                  	* java.lang.invoke.LambdaForm$MH+0x000000060104d400.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
0x910023e12a0003e2		????????
----------------- 2943659 -----------------

When setting -XX:+PreserveFramePointer we can walk them again (also tweaked the code to print the correct enterSpecial frame):

----------------- 2947348 -----------------
"ForkJoinPool-1-worker-2" #30 daemon prio=5 tid=0x0000ffff1c008040 nid=2947348 runnable [0x0000ffff7c8dd000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x0000ffffaee18b24	__clock_nanosleep + 0x84
0x0000ffffaee1dc0c	__GI___nanosleep + 0x1c
0x0000ffffaee1dacc	__sleep + 0x4c
0x0000ffff976faa38	<nep_invoker_blob>
0x0000ffff90806f3c	* java.lang.invoke.LambdaForm$MH+0x000007e001047800.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
0x0000ffff90bad354	* java.lang.invoke.LambdaForm$MH+0x000007e00104d400.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
0x0000ffff97597d60	* jdk.internal.foreign.abi.DowncallStub+0x000007e001048800.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x0000ffff908b7a70	* java.lang.invoke.LambdaForm$DMH+0x000007e001048c00.invokeStaticInit(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Compiled frame)
0x0000ffff90b9b710	* java.lang.invoke.LambdaForm$MH+0x000007e00104c800.invoke(java.lang.Object, int) bci:44 (Compiled frame)
0x0000ffff90b9041c	* java.lang.invoke.LambdaForm$MH+0x000007e00104c400.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Compiled frame)
0x0000ffff97597f30	* LingeredAppWithVirtualThread.run() bci:15 line:65 (Interpreted frame)
0x0000ffff975963b8	* jdk.internal.vm.Continuation.enterSpecial(jdk.internal.vm.Continuation, boolean, boolean) bci:0 (Compiled frame)
0x0000ffff97fa5fbc	* jdk.internal.vm.Continuation.run() bci:152 line:251 (Compiled frame)
0x0000ffff90b85888	* java.lang.VirtualThread.runContinuation() bci:100 line:293 (Compiled frame)
0x0000ffff97fa3eb0	* java.lang.VirtualThread$$Lambda+0x000007e001034c88.run() bci:4 (Compiled frame)
0x0000ffff90b8445c	* java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec() bci:4 line:1596 (Compiled frame)
0x0000ffff90b83298	* java.util.concurrent.ForkJoinTask.doExec() bci:10 line:511 (Compiled frame)
0x0000ffff90b81e2c	* java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) bci:5 line:1450 (Compiled frame)
                  	* java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) bci:364 line:2019 (Compiled frame)
0x0000ffff97fa03a8	* java.util.concurrent.ForkJoinWorkerThread.run() bci:31 line:187 (Compiled frame)
0x0000ffff9759349c	<StubRoutines (initial stubs)>
0x0000ffffad95020c	JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x45c
0x0000ffffad950880	JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x278
0x0000ffffad950e14	JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0x8c
0x0000ffffadaee66c	thread_entry(JavaThread*, JavaThread*) + 0xc4
0x0000ffffad98de68	JavaThread::thread_main_inner() + 0x108
0x0000ffffae301efc	Thread::call_run() + 0xac
0x0000ffffadfe105c	thread_native_entry(Thread*) + 0x12c
0x0000ffffaede3b50	start_thread + 0x300

I would have expected that mixed mode walked the stack similar to how we do it in the VM with NativeStackPrinter using frame::next_frame(), i.e we change how we get the sender based on the frame.

Anyways, I was just curious about the difference in output with mixed mode.

[1]

jdk/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/debugger/linux/aarch64/LinuxAARCH64CFrame.java

Line 56 in d6537c6

public CFrame sender(ThreadProxy thread) {

[2]

jdk/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/PStack.java

Line 148 in d6537c6

if (cb.isNMethod()) {

test/hotspot/jtreg/serviceability/sa/LingeredAppWithVirtualThread.java

YaSuenag · 2025-10-15T00:49:56Z

I guess jhsdb jstack --mixed might have potential bug(s) in -Xcomp regardless of continuation stub handling.

I ran jhsdb jstack --mixed with this PR to attach to the process running with -Xcomp on Linux x86_64, I got similar strange stack traces (without error) as following. It seems not to unwind completely in -Xcomp. LWP 1332 was failed to find sender frame from 0x7f9debcd96c0, LWP 1331 and 1330 are not compiled frame, but they couldn't find sender frame (maybe stack unwinder in jhsdb decide to stop unwinding because sender frame is NULL).

----------------- 1332 -----------------
"ForkJoinPool-1-worker-2" #29 daemon prio=5 tid=0x00007f9dfc2ad7b0 nid=1332 runnable [0x00007f9dd19fd000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x00007f9e0337e462      __syscall_cancel_arch + 0x32
0x00007f9e0337275c      __internal_syscall_cancel + 0x5c
0x00007f9e033cec37      __GI___nanosleep + 0x17
0x00007f9e033e114e      __sleep + 0x3e
0x00007f9deb4b361e      <nep_invoker_blob>
0x00007f9debcd96c0      * java.lang.invoke.LambdaForm$MH+0x000000000f047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
0x8b486428ec834853              ????????
----------------- 1331 -----------------
"ForkJoinPool-1-worker-1" #27 daemon prio=5 tid=0x00007f9dfc2ac5b0 nid=1331 runnable [0x00007f9dd1afd000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x00007f9e0337e462      __syscall_cancel_arch + 0x32
0x00007f9e0337275c      __internal_syscall_cancel + 0x5c
0x00007f9e033cec37      __GI___nanosleep + 0x17
0x00007f9e033e114e      __sleep + 0x3e
0x00007f9deb4b361e      <nep_invoker_blob>
0x00007f9de45d5c2c      * java.lang.invoke.LambdaForm$MH+0x000000000f047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
                        * java.lang.invoke.LambdaForm$MH+0x000000000f04f000.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
                        * jdk.internal.foreign.abi.DowncallStub+0x000000000f048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
----------------- 1330 -----------------
"VirtualThread-unblocker" #25 daemon prio=5 tid=0x00007f9dfc297b50 nid=1330 runnable [0x00007f9dd1bfd000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_blocked
0x00007f9e0337e462      __syscall_cancel_arch + 0x32
0x00007f9e0337275c      __internal_syscall_cancel + 0x5c
0x00007f9e0337549e      __GI___pthread_cond_wait + 0x14e
0x00007f9e01f0d834      PlatformEvent::park() + 0xe4
0x00007f9e019889ca      JVM_TakeVirtualThreadListToUnblock + 0x1ba
0x00007f9debc9cc16      java.lang.VirtualThread.takeVirtualThreadListToUnblock() + 0x96 (Native method)

In addition, I saw following error when I added both -Xcomp and -XX:+PreserveFramePointer in Linux x86_64. Note that I haven't seen this with -XX:+PreserveFramePointer only.

java.lang.RuntimeException: newSP(0x00007f345cb14690) is not above oldSP(0x00007f345cb14690)
        at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:156)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.initJFrameCache(PStack.java:224)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:73)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:65)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.PStack.run(PStack.java:60)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.run(JStack.java:67)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:278)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:241)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:134)
        at jdk.hotspot.agent/sun.jvm.hotspot.tools.JStack.runWithArgs(JStack.java:90)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJSTACK(SALauncher.java:306)
        at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:507)


----------------- 1425 -----------------
"ForkJoinPool-1-worker-4" #33 daemon prio=5 tid=0x00007f34782bac50 nid=1425 runnable [0x00007f345ca14000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x00007f3482592462      __syscall_cancel_arch + 0x32
0x00007f348258675c      __internal_syscall_cancel + 0x5c
0x00007f34825e2c37      __GI___nanosleep + 0x17
0x00007f34825f514e      __sleep + 0x3e
0x00007f34673a899e      <nep_invoker_blob>
0x00007f3467ce4d44      * java.lang.invoke.LambdaForm$MH+0x0000000034047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
0x00007f3467ce5094      * java.lang.invoke.LambdaForm$MH+0x000000003404e800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
0x00007f346733fe48      * jdk.internal.foreign.abi.DowncallStub+0x0000000034048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x00007f3460674744      * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Compiled frame)
0x00007f3460739944      * java.lang.invoke.LambdaForm$MH+0x000000003404c800.invoke(java.lang.Object, int) bci:44 (Compiled frame)
0x00007f346072f3ac      * java.lang.invoke.LambdaForm$MH+0x000000003404b800.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Compiled frame)
0x00007f346733fd00      * Test.run() bci:9 line:40 (Interpreted frame)
0x00007f346733e098      <StubRoutines (continuation stubs)>
0x00007f3467cbeb50      * jdk.internal.vm.Continuation.run() bci:152 line:251 (Compiled frame)
0x00007f3467cd0eb8      * java.lang.VirtualThread.runContinuation() bci:100 line:293 (Compiled frame)
0x00007f3467cbbf40      * java.lang.VirtualThread$$Lambda+0x000000003402a848.run() bci:4 (Compiled frame)
0x00007f3467cd13e8      * java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec() bci:4 line:1596 (Compiled frame)
0x00007f3467cd01e0      * java.util.concurrent.ForkJoinTask.doExec() bci:10 line:511 (Compiled frame)
                        * java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(java.util.concurrent.ForkJoinTask, int) bci:5 line:1450 (Compiled frame)
0x00007f346733fd00      * java.util.concurrent.ForkJoinPool.runWorker(java.util.concurrent.ForkJoinPool$WorkQueue) bci:364 line:2019 (Interpreted frame)
0x00007f3467cb6b04      * java.util.concurrent.ForkJoinWorkerThread.run() bci:31 line:187 (Compiled frame)
0x00007f34673386fd      <StubRoutines (initial stubs)>
0x00007f34809c0a7e      JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x4ce
0x00007f34809c11b3      JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x2d3
0x00007f34809c17bb      JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0xab
0x00007f3480b83a90      thread_entry(JavaThread*, JavaThread*) + 0xd0
0x00007f3480a02806      JavaThread::thread_main_inner() + 0x256
0x00007f3481665b57      Thread::call_run() + 0xb7
0x00007f34810fc588      thread_native_entry(Thread*) + 0x128
0x00007f3482589f54      start_thread + 0x2e4

----------------- 1424 -----------------
"ForkJoinPool-1-worker-3" #31 daemon prio=5 tid=0x00007f34782b6d00 nid=1424 runnable [0x00007f345cb14000]
   java.lang.Thread.State: RUNNABLE
   JavaThread state: _thread_in_native
0x00007f3482592462      __syscall_cancel_arch + 0x32
0x00007f348258675c      __internal_syscall_cancel + 0x5c
0x00007f34825e2c37      __GI___nanosleep + 0x17
0x00007f34825f514e      __sleep + 0x3e
0x00007f34673a899e      <nep_invoker_blob>
0x00007f3467ce4d44      * java.lang.invoke.LambdaForm$MH+0x0000000034047000.invoke(java.lang.Object, long, int) bci:10 (Compiled frame)
0x00007f346074ed84      * java.lang.invoke.LambdaForm$MH+0x000000003404e800.invokeExact_MT(java.lang.Object, long, int, java.lang.Object) bci:21 (Compiled frame)
0x00007f346733fe48      * jdk.internal.foreign.abi.DowncallStub+0x0000000034048000.invoke(java.lang.foreign.SegmentAllocator, java.lang.foreign.MemorySegment, int) bci:44 (Interpreted frame)
0x00007f3467cda978      * java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int) bci:14 (Compiled frame)
0x00007f3467cd9118      * java.lang.invoke.LambdaForm$MH+0x000000003404d000.invoke(java.lang.Object, int) bci:44 (Compiled frame)
0x00007f3467cd5c94      * java.lang.invoke.LambdaForm$MH+0x000000003404b800.invoke_MT(java.lang.Object, int, java.lang.Object) bci:18 (Compiled frame)
0x00007f34607129fc      * Test.run() bci:9 line:40 (Compiled frame)
0x00007f346733e098      <StubRoutines (continuation stubs)>
0x00007f3467cbeb50      <Unknown compiled code>
0x00007f3467cd0eb8      <Unknown compiled code>
0x00007f3467cbbf40      <Unknown compiled code>
0x00007f3460726924      <Unknown compiled code>
0x00007f3460723b24      <Unknown compiled code>
0x00007f346733fd00      <interpreter> return entry points
0x00007f3467cb6b04      <Unknown compiled code>
0x00007f34673386fd      <StubRoutines (initial stubs)>
0x00007f34809c0a7e      JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 0x4ce
0x00007f34809c11b3      JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 0x2d3
0x00007f34809c17bb      JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 0xab
0x00007f3480b83a90      thread_entry(JavaThread*, JavaThread*) + 0xd0
0x00007f3480a02806      JavaThread::thread_main_inner() + 0x256
0x00007f3481665b57      Thread::call_run() + 0xb7
0x00007f34810fc588      thread_native_entry(Thread*) + 0x128
0x00007f3482589f54      start_thread + 0x2e4

This is a problem in restoring Java frame, not native frame - native frame unwinding depends on DWARF in Linux x86_64, on FP ( X29 ) in AArch64. Thus we might need to fix unwinder for Java frames. But it is another issue.

plummercj

The changes look good. Thanks for taking care of this.

test/hotspot/jtreg/serviceability/sa/LingeredAppWithVirtualThread.java

pchilano

Looks good to me too, thanks for fixing this.

YaSuenag · 2025-10-16T12:43:39Z

Thanks everyone for involving this PR!

/integrate

openjdk · 2025-10-16T12:45:13Z

Going to push as commit 1653999.
Since your change was applied there have been 126 commits pushed to the master branch:

5fc3904: 8369491: Temporarily revert default TIMEOUT_FACTOR back to 4
ead35a7: 8358942: HttpClient adds Content-Length: 0 for a GET request with a BodyPublishers.noBody()
d6c122b: 8369982: ProblemList jdk/jfr/jvm/TestWaste.java
... and 123 more: https://git.openjdk.org/jdk/compare/c12f0e74a9acfaa91b537853143ec94f143fe316...master

Your commit was automatically rebased without conflicts.

openjdk · 2025-10-16T12:45:25Z

@YaSuenag Pushed as commit 1653999.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8369505: jhsdb jstack --mixed cannot handle continuation stub on Linu…

32b75fc

…x AMD64

openjdk bot added serviceability serviceability-dev@openjdk.org hotspot-runtime hotspot-runtime-dev@openjdk.org labels Oct 9, 2025

YaSuenag marked this pull request as ready for review October 9, 2025 23:30

openjdk bot added the rfr Pull request is ready for review label Oct 9, 2025

Fix trivial bug

d2a80f5

Fix for AArch64

2f3eee6

YaSuenag changed the title ~~8369505: jhsdb jstack --mixed cannot handle continuation stub on Linux AMD64~~ 8369505: jhsdb jstack --mixed cannot handle continuation stub on Linux Oct 11, 2025

Add testcase

915fca8

Apply changes to RISC-V code

57294d7

plummercj reviewed Oct 13, 2025

View reviewed changes

test/hotspot/jtreg/serviceability/sa/TestJhsdbJstackMixedWithVirtualThread.java Outdated Show resolved Hide resolved

YaSuenag changed the title ~~8369505: jhsdb jstack --mixed cannot handle continuation stub on Linux~~ 8369505: jhsdb jstack --mixed cannot handle continuation stub Oct 13, 2025

Update testcase

25d883f

plummercj suggested changes Oct 13, 2025

View reviewed changes

Update testcase

04d9309

YaSuenag changed the title ~~8369505: jhsdb jstack --mixed cannot handle continuation stub~~ 8369505: jhsdb jstack cannot handle continuation stub Oct 14, 2025

YaSuenag and others added 2 commits October 14, 2025 10:58

Switch argument of s(S)leep between Windows API and POSIX function

e8090fa

Update TestJhsdbJstackWithVirtualThread.java

b82bb9a

pchilano reviewed Oct 14, 2025

View reviewed changes

test/hotspot/jtreg/serviceability/sa/LingeredAppWithVirtualThread.java Show resolved Hide resolved

Move Thread.yield()

e64d696

plummercj approved these changes Oct 15, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 15, 2025

turbanoff reviewed Oct 15, 2025

View reviewed changes

test/hotspot/jtreg/serviceability/sa/LingeredAppWithVirtualThread.java Outdated Show resolved Hide resolved

turbanoff reviewed Oct 15, 2025

View reviewed changes

test/hotspot/jtreg/serviceability/sa/LingeredAppWithVirtualThread.java Outdated Show resolved Hide resolved

Cosmetic change

a8e3f4f

openjdk bot removed the ready Pull request is ready to be integrated label Oct 15, 2025

plummercj approved these changes Oct 15, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 15, 2025

pchilano approved these changes Oct 15, 2025

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Oct 16, 2025

openjdk bot closed this Oct 16, 2025

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 16, 2025

YaSuenag deleted the JDK-8369505 branch October 16, 2025 12:46

YaSuenag mentioned this pull request Oct 19, 2025

8370176: Mixed mode jhsdb jstack cannot unwind call stack with -Xcomp #27885

Open

3 tasks

8369505: jhsdb jstack cannot handle continuation stub #27728

8369505: jhsdb jstack cannot handle continuation stub #27728

Uh oh!

Conversation

YaSuenag commented Oct 9, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Warning

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Oct 9, 2025

Uh oh!

openjdk bot commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mlbridge bot commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

pchilano commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

plummercj commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

plummercj commented Oct 10, 2025

Uh oh!

plummercj commented Oct 10, 2025

Uh oh!

plummercj commented Oct 10, 2025

Uh oh!

pchilano commented Oct 10, 2025

Uh oh!

YaSuenag commented Oct 11, 2025

Uh oh!

YaSuenag commented Oct 11, 2025

Uh oh!

plummercj commented Oct 11, 2025

Uh oh!

YaSuenag commented Oct 11, 2025

Uh oh!

YaSuenag commented Oct 11, 2025

Uh oh!

RealFYang commented Oct 13, 2025

Uh oh!

YaSuenag commented Oct 13, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

YaSuenag commented Oct 14, 2025

Uh oh!

pchilano commented Oct 14, 2025

Uh oh!

Uh oh!

YaSuenag commented Oct 15, 2025

Uh oh!

plummercj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pchilano left a comment

Choose a reason for hiding this comment

Uh oh!

YaSuenag commented Oct 16, 2025

Uh oh!

openjdk bot commented Oct 16, 2025

Uh oh!

openjdk bot commented Oct 16, 2025

Uh oh!

Reviewers

Assignees

YaSuenag commented Oct 9, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Oct 9, 2025 •

edited

Loading

openjdk bot commented Oct 9, 2025 •

edited

Loading

mlbridge bot commented Oct 9, 2025 •

edited

Loading

pchilano commented Oct 10, 2025 •

edited

Loading

plummercj commented Oct 10, 2025 •

edited

Loading