VM instrumentation for third-party tools

[MikePall]

There are multiple third-party tools for instrumentation (profiling, debugging) of a running LuaJIT VM. Most of these have to rummage around in the VM internals, they only work with very specific LuaJIT versions and some even have to disassemble the code to gather some constants. This is an unfortunate state of affairs.

I'm willing to add some (near-) zero-overhead instrumentation to help these tools gather the required information.

This will be a low-level abstraction! The tool still needs to have some understanding of the VM internals: it needs to know how stack frames are linked, what constitutes a trace, and so on. Inevitably, this knowledge is still limited to a certain range of LuaJIT version. However, I'm aiming for some ABI-like guarantees that hold at least for a minor version, e.g. 2.1.*.

I'm aiming for cross-OS, cross-arch and cross-toolchain compatibility without adding a dependency on third-party instrumentation libraries:

The idea is to provide some exported symbols pointing to constants and arrays of tiny lookup functions. Machine code patch points (a nop) inside certain VM functions can be added, too, if desired.

This is a call for all tool authors to tell me:

• What kind of static information do you need from a linked LuaJIT library?
• What kind of runtime information do you need from a running LuaJIT VM instance?

Please give both the 'why' and the 'what', so we can make an informed decision on the best way to provide you with this information. E.g. 'We want to show VM trace details in a stack backtrace' + 'We need to map a machine code address to a trace number' + 'We need to get specific infos for a trace, given its number'.

Please specify the name of the tool, a link to your existing code and an exhaustive list of the whys and whats. Please also point out which alternative approaches would work for you in each case, e.g. constant field offset vs. lookup function.

I'll gather and organize everything in this issue, after a sufficient number of responses came in.

[GitSparTV]

Hi! We are currently working on adding support for LuaJIT too. We use similar approach to openresty, but we advanced in some points.

As a lead of this feature, I'm happy to share my feedback. We are still WIP and plan to ship truncated (we researched more deeper, but maintainers asked to make PR smaller) limited MVP soon. Current state can be found here (mainly perforator/agent/collector/progs/unwinder/lua/unwind.h).

When we talk about generic profiler, this means that relying on globalL is not an option because 99% of the time people will profile an embedded version of LuaJIT. In BPF profiler we use perf event, which calls our code and provides us with current registers.

What we are doing?

1. Find global_State. Note: we assume process has a single GG_State. No multithreaded/multistate shenanigans.
   • First we need to catch when rip will be inside a LuaJIT binary. This part is on us. We provide BPF with that info. (optional, but helps to reduce CPU usage)
   • If we are in LuaJIT binary chances are r14 register is a dispatch. Here is where first offsets are required: we do r14 - GG_G2DISP and check if G(possible_global_state->cur_L) == possible_global_state. Requirements: GG_G2DISP and offsetof(global_State, cur_L). Currently we assume lua_State members can't vary from build to build. (I'm happy to be wrong, but I would like to have flags/config/compiler where these offsets will be different)
   • We disassemble few public Lua C API function to get these offsets.
   • Save found state
2. Once global_State found (and checked if it still valid by G <-> L cross-checking) start walking the stack
   • Check current vmstate. If it's running a trace:
     • Get current trace and save its mcode and szmcode (G2J(g)->trace[tracen]). This is required to help native stack unwinder get stack frames below JIT-trace frame. See bullet 4 for more info.
     • Get g->jit_base instead of L->base
   • We use L->base, unless we are currently executing some of lj_BC_ instruction. And also a special case with lj_BC_RET: during its execution the base is only about to change. Such hacks are required because we want to extract the current frame as well!
   • Init from frame = nextframe = base - 1 and let's go until frame > bottom
     • Skip dummy and vararg pseudo frames
     • I've experimented with determining an approximate stack size but still not sure if it's a useful guard:

       int stack_size = (L->maxstack - tvref(L->stack)) >> 3;
       int free_slots = (L->maxstack - L->top - 8) >> 3;
       
       int approx_size = stack_size - free_slots;

3. For each frame:
   • As a sanity check if tvisfunc(frame - LJFR2) is true.
   • For lua frames: get proto->firstline, proto->numline, proto->funcname, proto->chunkname.
     • Our MVP will stop here, but in reality we already wrote code to get real func name and actual line number from the caller. This is so fun especially when you can't use while loops in BPF (talking to lj_debug_slotname) 🙂
   • For C/FF frames: get fn->c.f and c.ffid
     • We don't need any info about the function name. We rely on debug info that is used on frame post-processing. But if there is no info, the only thing we can do is a hardcoded list of FF function names :-)
4. Native unwinder fix (currently in research)
   • The profiler we are contributing in has DWARF unwinding support with FP as a fallback. When it encounters a JIT frame it stops because: there is no DWARF info, attempting to get next frame by FP failed.
   • LuaJIT's JIT frames at the first glance have constant offset CFRAME_SIZE_JIT, but by looking at machine code or by existence of tr->spadjust or by lj_gdbjit code, I see it's not that simple. So... the simplest solution was to take L->cframe, add CFRAME_SIZE and take 2 uint64_t values behind.
   • When unwinder falls into tr->mcode >= rip > tr->mcode + tr->szmcode range we apply this fix and move on.
5. Frame merging
   • Currently to merge native frames we look for hardcoded list of public Lua C API functions such as lua_pcall, lua_cpcall and others. I don't like this approach because some functions like lua_lessthan or lua_equal call VM conditionally. So instead we extract debug information about lj_vm. It has very unique FDE with size 12k+ bytes :-). Using VM range we can determine if a native frame was executing LuaJIT VM even without debug symbols being present. In theory we could use GG->dispatch but it's just more work.
   • The remaining logic is easy: get first VM call, insert Lua frame there. If we called a C function, use that as a sentinel and insert next frames after it.

I think that's it for now.
As a reference, you can look at Python unwinding. I can't find the commit, but recently Python added some helpers that can be obtained through public symbols with all useful offsets and flags

All assumptions

• x86-64 Linux currently
• LJ_FR2, LJ_GC64, with unwind
• We plan to support versions starting from beta3 up to current HEAD.
• One GG_State in the process
• Not testing coroutines yet, but I assume they work ok
• lua_State and other GC members are the same everywhere, so we can use their offsets from luajit headers we ship (please break me)
• offsetof(global_State, vmstate) == 184
• offsetof(GCtrace, code) == 88
• offsetof(GCtrace, szcode) == 84
• jit_base is next to cur_L (TODO for us: find what public symbol can provide us with the offset)
• CFRAME_SIZE value is what headers tell
• Hardcoded list of fast function names.

Extracted info

• GG_G2DISP
• offsetof(global_State, cur_L)
• LuaJIT VM range (for frame order on flamegraph)
• G2J
• offsetof(jit_State, trace)

A global symbol with a struct will be great in 3.0.
Constants like LJ_FR2, LJ_GC64, CFRAME_SIZE. Everything that can affect offsets to critical structs.

[rreive]

OK a little help from AI here to document what this is all about... and why it's really important.

Developer Advisory: Enhancing LuaJIT Observability with VM Instrumentation for Third-Party Tools (e.g., eBPF/Perf Integration)

Summary Preface- "The story why this proposal is important"
Imagine you're the captain of a high-speed cargo ship carrying sensitive, encrypted goods across stormy seas (your QUIC HTTP/3 data flows) while also securing treasures in fortified vaults on board (your local encrypted storage). The ship (your LuaJIT-powered application) is incredibly fast and efficient, but in rough conditions or during an emergency, you need to see exactly what's happening inside the engine room, on the bridge, and in every cargo hold—without slowing the ship down or opening the vaults.
Traditional "black box" LuaJIT runs fast but hides its internal workings, making it hard to diagnose issues, optimize performance, or ensure compliance in demanding environments. The proposed VM instrumentation (GitHub Issue #1457) is like installing a sophisticated, non-intrusive monitoring system with cameras, sensors, and real-time dashboards. It lets external tools (like eBPF profilers) peek inside safely, providing clear views of Lua call stacks, JIT-compiled code behavior, VM states, and more. This empowers developers to maintain top speed, security, and reliability—especially for lightweight, headless, FIPS-compliant systems handling encrypted data in transit and at rest.

Core Concept: LuaJIT VM Instrumentation for Third-Party Observability Tools
The enhancement focuses on adding lightweight, optional instrumentation points/hooks in the LuaJIT VM. These enable seamless integration with modern observability tools (perf, eBPF/BPF profilers, system tracers) without significant runtime overhead when disabled. Key goals include better stack unwinding, JIT trace visibility, and VM state exposure for profiling, tracing, and debugging.
Key Capabilities and Features

<style type="text/css"></style> Feature | Description | Technical Details -- | -- | -- Stack Unwinding Support | Allows external tools to accurately reconstruct mixed native + Lua + JIT call stacks. | Frame pointers, metadata for LuaJIT internals (L->base, ci, etc.), JIT code annotations. JIT Trace Visibility | Exposes compiled trace boundaries and execution. | Markers/hotspots for perf/eBPF to attribute time to specific Lua functions/traces. VM State Instrumentation | Hooks for GC, interpreter loops, JIT compilation events. | Low-overhead tracepoints or probe-friendly locations. Zero/ Low Overhead Mode | Instrumentation is compile-time or runtime optional. | Disabled by default; minimal impact on production performance. eBPF Compatibility | Direct support for kernel/user-space eBPF unwinders. | Enables zero-instrumentation kernel-level profiling of LuaJIT apps.

Advantages and Benefits

Performance Insight Without Penalty: Profile production workloads (e.g., high-throughput QUIC handlers) with negligible overhead.

Security & Compliance Alignment: Observe behavior in FIPS-compliant encrypted flows without decrypting payloads or adding risky code.

Debugging Complex Flows: Mixed Lua/C/JIT stacks become transparent for troubleshooting latency in UDP-based data pipelines or storage ops.

Ecosystem Integration: Works with tools like OpenTelemetry eBPF profiler, Parca, BCC/bpftrace, system perf, and custom LuaJIT-aware tracers.

Lightweight Headless Suitability: Ideal for containerized/embedded services—no GUI, minimal resource use.

Benefits in Context:

Primary Use Case (High-volume QUIC/HTTP/3 encrypted data flows): Real-time latency profiling of Lua handlers processing in-transit data. Identify JIT trace hot spots or GC pauses affecting UDP throughput.
Secondary Use Case (Local at-rest encrypted storage): Monitor I/O-bound Lua scripts for storage encryption/decryption ops, ensuring FIPS compliance while optimizing disk/CPU usage.
Exception Use Cases (Errors, anomalies, audits): On-demand tracing for crashes, unexpected latency spikes, or security audits—capture full stack context including Lua frames without restarting the service.

ASCII Diagrams

1. High-Level Architecture (Observability Integration)

[External Tools: eBPF Profiler / perf / bpftrace]
|
v (Probes / uprobes on LuaJIT symbols)
[Host OS Kernel / User Space]
|
+---------------------------------------+
| LuaJIT VM (Instrumented) |
| - Interpreter Loop |
| - JIT Compiler |
| - Lua Stack (L->ci) |
| - Native Frames |
+--------------------------------------+
|
Mixed Stacks: [Native] -> [JIT Trace] -> [Lua C Function] -> [Lua Script]

2. Stack Unwinding Flow (Simplified)

CPU Sample (eBPF/perf)
|
Probe hits LuaJIT frame
|
Check for LuaJIT magic / L (lua_State *)
|
IF Lua frame:
Unwind Lua stack: L->ci -> func -> pc -> line (via debug info)

• Native frames above/below
  ELSE:
  Standard DWARF / frame pointer unwind
  |
  Output: Flamegraph / Trace with Lua symbols + JIT names

3. Use Case Data Flow with Observability

QUIC/HTTP/3 Listener (UDP)
|
LuaJIT Handler (Encrypted in-transit)
|--> [Instrumented VM] --> eBPF Samples
| (JIT execution, GC, I/O)
|
Local FIPS Storage (Encrypted at-rest)
|
Observability: Low-overhead metrics + full stacks on anomaly

Applicability to Encrypted Data Contexts
In lightweight headless setups:

In-Transit (QUIC/HTTP/3 over UDP): Instrumentation helps profile packet processing loops without intercepting encrypted payloads. Benefits include identifying bottlenecks in Lua-based protocol logic while maintaining NIST FIPS compliance.
At-Rest Storage: Monitor encryption wrappers and storage ops for performance tuning and anomaly detection (e.g., unusual GC during bulk writes).
Hybrid Flows: End-to-end visibility across network + storage layers supports reliability in secure, resource-constrained environments.

Instrumentation ensures observability scales with LuaJIT's strengths (speed, small footprint) rather than compromising them.

Appendix: Well-Documented LuaJIT VM-Relevant Code Example with eBPF Stack Unwinding Logic

Note: This is a conceptual/example implementation inspired by community efforts (e.g., nginx-lua-ebpf, OpenTelemetry eBPF profiler for LuaJIT). Actual integration depends on the outcome of Issue #1457. Test in a controlled environment.

Example eBPF Side (Simplified C for bpftrace/BCC-style unwinder)

C

// bpf_unwinder.bpf.c (excerpt)
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>

// Assume LuaJIT lua_State* detection via uprobe on key functions
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 10240);
__type(key, u64); // pid + tid or similar
__type(value, u64); // lua_State pointer
} lua_states SEC(".maps");

SEC("uprobe/libluajit:lj_BC_xxx") // Example probe on bytecode dispatch
int probe_luajit_frame(struct pt_regs ctx) {
u64 luaL = (u64)PT_REGS_PARM1(ctx); // Example: lua_State
if (luaL) {
// Store or signal user-space with state
u64 key = bpf_get_current_pid_tgid();
bpf_map_update_elem(&lua_states, &key, &luaL, BPF_ANY);
}
return 0;
}

// Stack walker helper (user-space or more advanced BPF)

LuaJIT / C Side Instrumentation Example (for better support)

lua

-- example_observability.lua
local ffi = require("ffi")

-- Hypothetical instrumentation API post-#1457
local lj = ffi.load("luajit") -- or internal hooks

ffi.cdef[[
// Example VM hook registration
typedef void (trace_cb)(const char event, void* data);
void lj_register_observability_hook(trace_cb cb, int flags);
]]

local function observability_hook(event, data)
print("VM Event:", event) -- Log or forward to eBPF map
-- e.g., "jit_trace_start", "gc_pause", "lua_call"
end

-- Register (low overhead)
lj.lj_register_observability_hook(observability_hook, 0x01) -- Flags for specific events

-- Your app code
local function handle_quic_request(data)
-- Process encrypted flow...
-- Storage op...
end

User-Space Stack Unwinding Logic (Python/Pseudocode with eBPF)

Python

unwinder.py (conceptual)

def unwind_luajit_stack(lua_state_ptr, sample_ip):
frames = []
# Walk LuaJIT call info chain (ci)
ci = get_lj_ci(lua_state_ptr) # From memory read or metadata
while ci:
func = get_lj_func(ci)
pc = get_lj_pc(ci)
line_info = get_debug_info(func, pc) # Source line
frames.append(f"Lua: {get_func_name(func)}:{line_info}")
ci = ci.previous
# Merge with native frames from perf/eBPF
native_frames = unwind_native(sample_ip)
return native_frames + frames[::-1] # Combined stack

Usage Notes:

Compile LuaJIT with instrumentation flags (e.g., -DLUAJIT_ENABLE_OBSERVABILITY).
Use tools like bpftrace or OpenTelemetry eBPF for zero-code-change profiling.
For FIPS/secure environments: Ensure hooks do not expose plaintext data.
Test stack accuracy on your specific LuaJIT version (2.1+ recommended).

This advisory provides a foundation. Monitor the GitHub issue for official implementation details and contribute if possible. For production use, validate performance impact under load.

[gnurizen]

Our LuaJIT unwinder is roughly documented here:

https://github.com/parca-dev/opentelemetry-ebpf-profiler/blob/main/interpreter/luajit/README.md

It supports x86_64 and arm64 and is implemented as a mix of eBPF and Go code doing privileged "remote" (ie syscall) reads of the lua processes memory. Customers have been using it for about a year. It requires zero instrumentation and works with every form of luajit we've come across.

We are in the process of upstreaming it to https://github.com/open-telemetry/opentelemetry-ebpf-profiler.

We currently extract three pieces of the information from the luajit code (using handcrafted slightly fragile disassembly of luajit executable):

1. The distance from g to dispatch
2. The distance from g to J.trace
3. The offset of cur_L in global_State

At runtime we use some heuristics to try to find "L" that aren't perfect and can sometimes take a while to "hit" on heavily JIT'd code (although that's usually just synthetic microbenchmarks, real world code tends to work better).

Our code also has baked in knowledge of the layout of GCproto and GCtrace and how to pull traces out of the jitState.

It also knows the luajit bytecode format and uses that knowledge to walk luajit functions and extract "names" for functions similar to how luajits built in profiler does it.

We would be happy with some USDT probes baked into the executable to get the offsets as a start, we want to support stripped luajit binaries which some customers build themselves so we don't want to rely on symbol resolution.

Next up would be a more reliable way to get L/G, there are lots of JIT situations where the r14 dispatch register trick doesn't work.

If there were probes that we could hook into to learn about trace creation and trace destruction that would be nice but we want to preserve the ability to hook into and profile a running luajit process with existing traces in play, we just need to know the extents of the mcode and spadjust.

[rreive]

Here is the public link on Grok for the docs suggested per this issue, have at er.., my day job is in the way ;)

https://grok.com/share/c2hhcmQtMg_751d337b-b4e5-4161-8322-a86cb820b949

[artemking4]

I was actually researching how you would make a profiler that would mix stack information for native/luajit/other languages in one callstack a little while ago, and was met with an issue of not really being able to identify where a luajit interpreter frame begins, and then, getting its states. I guess you could just try a bunch of stuff to detect a frame, decode the cframe and check its validity, but that's very unportable and just hard to work with.

[gnurizen]

@artemking4 that's a good point, I forgot about this, the hacky solution we came up with was to guess which FDE was the interpreter by looking at its size in the eh_frame data, see: https://github.com/parca-dev/opentelemetry-ebpf-profiler/blob/main/interpreter/luajit/luajit.go#L171

Its a monster compared to normal functions so it sticks out but this is totally a hack and something to tell us this w/o guessing would be nice.

[gnurizen]

An elegant way to get the "G/L" context pointers w/o stack/register discovery guess work would be to have some kind of heartbeat USDT probe, like once a second would be enough. Maybe it could be attached to a GC work check routine?

[artemking4]

IMO the best approach, at least, for profilers, would be a certain API that would tell you what global lua states are in a certain thread's stack, and then an API to decode their cframe and lua stack in an ABI-stable manner.

Sounds very much doable with clever use of thread local storage, and that wouldnt require any workarounds.

Maybe also an api similar to sethook, but closer to how the profiling callback runs, so that users could trigger their own hooks that would abort JITted code and so on.

As Spar has already said, a global symbol with the most crucial definitions would be very helpful, but dealing with the plethora of different states the vm can be at for an outsider is very troubling. So some user-friendly apis would be greatly appreciated

[gnurizen]

Its not clear to me how a continuous profiler meant for production like Parca would take advantage of profiling "APIs". We don't want to interact with the program at all, we want it to execute exactly as it would if the profiler wasn't there and none of our profiler code executes in the processes we're profiling. USDT probes as eBPF attachment points provide a nice mechanism for a read-only, hands off kernel orchestrated introspection with near zero overhead which is why I keep bringing them up. This may be helpful: https://www.polarsignals.com/blog/posts/2025/12/10/usdt-deep-dive

[MikePall]

It looks like we want to support two different kinds of tools:

In-process tools: running in the same process as the LuaJIT VM. But not necessarily in the same thread. Examples would be the built-in LuaJIT profiler, a memory profiler, a cross-language sampling profiler or a crash reporter.

Out-of-process tools: running in a separate process. Often system-wide, assisted by OS tracing infrastructure. Examples would be Parca or Perforator.

The needs seem to be quite different, but I think we can cater to both of them.

In-process tools can call helper functions provided by the VM:

First, they need to find some kind of anchor (g and/or L) and execution context. Either via explicit invocation or indirectly from the CPU state. Then they can gather VM info, traverse frames and so on.

The most accessible way would be to provide a global symbol that points to some helper functions that do all of the work and hide most of the VM implementation details.

Out-of-process tools rely on some kind of OS-wide tracing infrastructure:

Either like SystemTap, DTrace, eBPF and USDT probes: the executable code is patched at run-time to invoke the tracing functionality, but only when enabled. Or like Windows ETW: these are explicit calls, but they do nothing when not enabled.

Out-of-process tools generally cannot call helper functions in the traced process. The VM should explicitly feed the anchor/context to the probes, otherwise the tools need to guess (which is what you're currently doing).

The tools also need internal implementation details about the VM to derive the needed info. The VM could help somewhat and provide a few constants and runtime variables, but not in-process helper functions.

I think there are two main difficulties:

1. How to find the anchor/context given the current process state?

   It looks like the bare minimum required input would be the CPU registers of an unwound C stack frame. And a VM-specific function that resolves this to an anchor and an execution context. Plus a (fast) way to find out whether a frame is actually provided by the VM.

   OTOH, for certain tools, like non-sampling profilers, it would be easier to get explicitly called with the anchor/context.

2. How to hide VM implementation details?

   Helper functions can certainly be provided for some common tasks, but anything beyond that requires precise VM implementation details. And out-of-process tools are out of luck (*).

(*) I think this means there's a need for some kind of language- or VM-specific plug-in that you could easily load into your agent controllers. This would hide all/most of the implementation-specific work, like enumerating internal VM frames. It seems pointless and fragile to duplicate more or less the same code for every language and VM version in every tool. May I suggest the relevant parties get together, find common required functionality and define such a plug-in infrastructure? Then every language runtime could provide this plug-in along with its assets, e.g. a separate shared library, eBPF code in a special ELF section, or whatever would be convenient for you.

[GitSparTV]

Also forgot to mention. tarantool has gdb/lldb scripts to inspect lua state. Not like it's a complete debugger, but it worth mentioning because you can see how many stuff have to be reimplemented to show output like debug.traceback + debug.getlocal:

Description: tarantool#9
Code: https://github.com/GitSparTV/tarantool_luajit/blob/3b8578c26910db7fd26c8d046efb29a054094b7b/src/luajit_lldb.py
The reason I had to manually rewrite some debug internals is because they were static in code. Optimizer reordered parameters or even inlined them. They can't be called from debugger anymore

I've enhanced their script in parallel. For perforator we want to make sure that on any point/rip we can correctly extract stack and function names. So contributing here was worth it.

There are other debuggers, for example, on VSCode marketplace. Haven't looked at their implementation. I suppose someone still uses debug.sethook