Support JIT-compiled code debugging for Release builds

[c0d1f1ed]

The current support for debugging JIT-compiled code depends on GDB or LLDB to be able to observe the __jit_debug_register_code() function and __jit_debug_descriptor data structure. This only works when LLVM was built using the Debug or RelWithDebInfo configuration.

It would be great if it could also be made to work with a Release build, which would dramatically reduce the binary size. The JIT of a project I'm working on is 45 MB for a Release build, but over 1 GB for a RelWithDebInfo build.

The above two symbols are both defined in the llvm/lib/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.cpp file, so in theory it should suffice to generate debug info for just that file, and ensure it gets preserved during linking and doesn't get stripped afterwards. Alternatively if we had a build flag to optionally not define them in this file but just declare them extern, a project which links in LLVM's JIT can define them in one of its own source files and ensure they're available in the debug info.

Does either of these options sound reasonable, or is there a better way to handle this?

[llvmbot]

@llvm/issue-subscribers-orcjit

[dwblaikie]

Attributing the variable and function with __attribute__((used)) should suffice to preserve them through optimization, I think?

[dwblaikie]

(hmm, is the noinline insufficient? What actual problems are happening under optimization? It's possible that inline-like behavior is happening without the inliner (some interprocedural optimizations amount to the same thing (prove the function does nothing, remove do-nothing function calls, for instance - technically not inlining)) - in which case /maybe/ __attribute__((optnone)) might help, but even that may be insufficient, since it doesn't disable all interprocedural optimizations, though I think it should - there was a discussion a year or two ago about figuring out/fixing that... "noipa": https://discourse.llvm.org/t/revisiting-refining-the-definition-of-optnone-with-interprocedural-transformations/58095/2 )

[weliveindetail]

In my experience these symbols don't get dropped in Release builds. Double-checked on my Linux workstation just now:

> cat CMakeCache.txt | grep BUILD_TYPE
CMAKE_BUILD_TYPE:STRING=Release
> ninja llvm-jitlink-executor
> nm bin/llvm-jitlink-executor | grep __jit_debug
0000000000196030 D __jit_debug_descriptor
0000000000049b80 T __jit_debug_register_code

Maybe it's due to another configuration difference? Things can get a little more complicated, e.g. with shared libraries. I wrote a blog post with a troubleshooting section a while ago: https://weliveindetail.github.io/blog/post/2022/11/27/gdb-jit-interface-101.html#4-can-lldb-see-our-jit-interface-symbols

[lhames]

I haven't seen this issue either.

Does either of these options sound reasonable, or is there a better way to handle this?

Ideally we should define these in LLVM. It'd be awkward to require clients to define them for the sake of release builds.

What debugger are you seeing the issue with? And do the __jit_debug* symbols show up in your executable / library under nm or nm -D?

[c0d1f1ed]

Thanks everyone for the great pieces of information! I had previously assumed that without a .debug_info section the debugger wouldn't be able to put a breakpoint at __jit_debug_register_code() or read data from __jit_debug_descriptor, but that turns out to be incorrect. Indeed running strip --strip-all --keep-symbol=__jit_debug_register_code --keep_symbol=__jit_debug_descriptor on my RelWithDebInfo build still retained JIT debugging support. For the Release build I simply had a bug where it would inadvertently disable generating debug info for the JITed code. 🤠

I also got it working on Windows (our main platform). It already worked fine for the RelWithDebInfo build when using LLVM_DEFAULT_TARGET_TRIPLE=x86_64_pc-linux-elf (*), but it produced a huge .pdb file. For a Release build there's no .pdb and thus no symbols whatsoever...

But when I build my code with /Zi, link with /DEBUG, and declare both __jit_debug_register_code() and __jit_debug_descriptor as extern in my code (i.e. the inverse of what I proposed in my original message), a small .pdb gets produced and I'm able to debug the JITed code.

I'll double-check everything and close this if there's no LLVM-side code/build change required. Thanks again folks!

(*) There's probably a better way to produce object code in ELF format on Windows but I haven't looked into it yet.

[c0d1f1ed]

NVIDIA Warp now includes the changes to support debugging of JIT-compiled C++ code, all in one small package: pip install warp-lang. Thanks folks for the useful pointers, and the awesome LLVM project! 💯

[lhames]

Very cool!