Async Profiler and Java Flight Recorder are missing some methods at the bottom of the stack.

[qsLI]

From a code perspective, the correct call order should be： Thread.run -> FastThreadLocalRunnable.run -> ThreadPoolExecutor$Worker.run -> ThreadPoolExecutor.runWorker

✅using jstack:

❌using async profiler:

❌using jfr ExecutionSample event:

This doesn't seem to be a bug with async profiler, any thoughts?

[franz1981]

Did you added DebugNonSafepoints as suggested in the readme?

[qsLI]

Did you added DebugNonSafepoints as suggested in the readme?

I'll give it a try.

[heifrank]

Did you added DebugNonSafepoints as suggested in the readme?

@franz1981 If it's really DebugNonSafepoints problem, then jstack will also show missing stack frame, however jstack shows correctly.

[franz1981]

Depends. If there is a safepoint poll within an inlined method, jstack can still observe (and blame it), while, without the option I've specified, both jfr and async profiler won't be able to do the same (loop striping can still add polls to counted loops).
Regarding your method: are you using CPU profiling? If yes, jstack has captured a WAITING thread which is not something a CPU profiler would show up, because...it doesn't consume CPU (unless in the precess of contending on some process futex to do it).

[apangin]

If it's not an async-profiler bug, will it be better to submit an OpenJDK issue instead?
If you could provide a reproducible example, I'd be happy to analyze that.
Btw, does the issue happen with other JDK versions or with a different CPU architecture?

[qsLI]

If it's not an async-profiler bug, will it be better to submit an OpenJDK issue instead? If you could provide a reproducible example, I'd be happy to analyze that. Btw, does the issue happen with other JDK versions or with a different CPU architecture?

The issue cannot be stably reproduced at present, it only occurs occasionally. This case was found on an internal version ported based on OpenJDK 11.

[apangin]

How does --cstack vm work? Does it show missing frame?

[qsLI]

How does --cstack vm work? Does it show missing frame?

jdk.ActiveSetting cstack value is fp when using --cstack vm，seems a bug？

https://github.com/async-profiler/async-profiler/blob/29b78f4670b58272ce5968774baed06381f8d0aa/src/flightRecorder.cpp#L71C33-L71C33

using command:
./asprof -e cpu --cstack vm -d 60 -f p.jfr

jfr info:
jdk.ActiveSetting { startTime = 08:40:06.344 id = 107 name = "cstack" value = "fp" eventThread = "Attach Listener" (javaThreadId = 2700) }

[apangin]

jdk.ActiveSetting cstack value is fp when using --cstack vm，seems a bug？

Yes, thank you for pointing out. I pushed the fix.

[qsLI]

How does --cstack vm work? Does it show missing frame?

still not working with async-profiler-2.10-linux-x64

cmd ./asprof -e cpu --cstack vm -d 60 -f p.html

[apangin]

This does not look like a vm stack trace.
cstack=vm option appeared in async-profiler 3.0-ea.

[qsLI]

This does not look like a vm stack trace. cstack=vm option appeared in async-profiler 3.0-ea.

• cstack=vm mode is correct

• the same host with cstack=fp

[apangin]

Thank you for the update. Good to know async-profiler's own stack walker works correctly. It will eventually replace JVM's AsyncGetCallTrace and become the default.

[qsLI]

Thank you for the update. Good to know async-profiler's own stack walker works correctly. It will eventually replace JVM's AsyncGetCallTrace and become the default.

cstack=vm reads the symbol information (gHotSpotVMStructs) exported by the JVM, then finds the offset related to PCDesc, reads the corresponding data, and obtains the call stack information. Since the vm mode is correct, it can be inferred that the information of PCDesc is correct or enough, which has nothing to do with DebugNonSafePoint?

[apangin]

Yes, DebugNonSafepoints option may affect accuracy of the topmost virtual frames, but not a frame at the bottom of the stack.

[apangin]

Let me close the issue, since there is nothing to do on async-profiler side.