Skip to content

feat(instr): Phase 3 — INVOKEDYNAMIC handler isolation#830

Draft
jbachorik wants to merge 7 commits intodevelopfrom
phase3-invokedynamic-dispatch
Draft

feat(instr): Phase 3 — INVOKEDYNAMIC handler isolation#830
jbachorik wants to merge 7 commits intodevelopfrom
phase3-invokedynamic-dispatch

Conversation

@jbachorik
Copy link
Copy Markdown
Collaborator

@jbachorik jbachorik commented Apr 11, 2026

Summary

Replaces INVOKESTATIC handler copying with INVOKEDYNAMIC dispatch so probe handler methods stay in the probe class (bootstrap CL) and are called via ConstantCallSite, eliminating bytecode copying into target classes. Works on Java 8+.

Dispatch chain

InstrumentedMethod → INVOKEDYNAMIC → IndyDispatcher.bootstrap()
  → HandlerRepositoryImpl.resolveHandler()
  → MethodHandles.publicLookup().findStatic()
  → ConstantCallSite

New files

  • IndyDispatcher — Java 8+ bootstrap; resolves probe handler via publicLookup().findStatic() on the probe class; wired to HandlerRepositoryImpl via a volatile HandlerRepository repository bridge field set from the agent CL via reflection
  • HandlerRepositoryImpl (rewritten) — ConcurrentHashMap with UNRESOLVED sentinel for failed lookups; registerProbe() evicts stale UNRESOLVED entries to handle registration races
  • DispatchBenchmark — JMH benchmark comparing direct call baseline vs ConstantCallSite dispatch

Deleted

  • Indy.java — Java 15-specific defineHiddenClass bootstrap, no longer needed
  • CopyingVisitor.java — handler bytecode copying, no longer needed
  • static/ golden files (~198) — replaced by unified dynamic/ files

Refactored

  • Instrumentor.invokeBTraceAction — always emits INVOKEDYNAMIC; removed useHiddenClasses dual-mode gate; bootstrap handle owner: IndyIndyDispatcher
  • Probe lifecycle symmetry: registerProbe moved to BTraceProbeNode/BTraceProbePersisted.register() (after defineClass); unregisterProbe added to both unregister() methods; removed premature registration in BTraceProbeFactory and redundant cleanup in Client.onExit
  • BTraceRuntimeImpl_9/_11 — StackWalker frames filtered for org.openjdk.btrace.runtime.auxiliary.* in getCallerClassLoader() and getCallerClass()
  • Cushion-method infrastructure removed from BTraceClassWriter, BTraceTransformer, and Instrumentor

Test plan

  • ./gradlew :btrace-instr:test — all instrumentor tests pass
  • ./gradlew :btrace-instr:test -PupdateTestData — 382 dynamic golden files regenerated
  • ./gradlew :integration-tests:test -Pintegration — requires Docker (run separately)
  • ./gradlew :benchmarks:runtime-benchmarks:jmh -PjmhInclude=DispatchBenchmark

Acceptance criteria

  • IndyDispatcher works from Java 8+
  • ~400 redundant static/ golden files replaced with unified dynamic/ files
  • HandlerRepositoryImplTest verifies attach/detach cycle clears all handler cache entries
  • HandlerRepositoryImpl uses ConcurrentHashMap with UNRESOLVED sentinel
  • All instrumentor tests pass
  • DispatchBenchmark exists

🤖 Generated with Claude Code


This change is Reviewable

jbachorik and others added 2 commits April 11, 2026 16:36
Replace INVOKESTATIC handler copying with INVOKEDYNAMIC dispatch so probe
handler methods stay in the probe class (bootstrap CL) and are called via
ConstantCallSite, eliminating bytecode copying into target classes.

New:
- IndyDispatcher: Java 8+ bootstrap method using publicLookup().findStatic()
  on the probe class; wired from HandlerRepositoryImpl's static initializer
  via reflection on IndyDispatcher.repository (volatile bridge field)
- HandlerRepositoryImpl: rewrites resolveHandler() to return MethodHandle
  instead of byte[]; uses ConcurrentHashMap with UNRESOLVED sentinel for
  failed lookups; registerProbe() now evicts stale UNRESOLVED entries so
  late-resolving probes work after a registration race
- DispatchBenchmark: JMH benchmark for ConstantCallSite dispatch overhead

Deleted:
- Indy.java (Java-15-specific defineHiddenClass bootstrap, no longer needed)
- CopyingVisitor.java (handler bytecode copying, no longer needed)
- static/ golden files (~198 files, replaced by unified dynamic/ files)

Refactored:
- Instrumentor.invokeBTraceAction: always emits INVOKEDYNAMIC; removed
  useHiddenClasses dual-mode gate; bootstrap handle owner: Indy → IndyDispatcher
- Probe lifecycle symmetry: registerProbe() moved to BTraceProbeNode/
  BTraceProbePersisted.register() (after defineClass); unregisterProbe()
  moved to both unregister() methods; removed premature registration in
  BTraceProbeFactory and redundant unregisterProbe in Client.onExit()
- BTraceRuntimeImpl_9/_11: StackWalker frames filtered for
  org.openjdk.btrace.runtime.auxiliary.* to skip IndyDispatcher frames in
  getCallerClassLoader() and getCallerClass()
- HandlerRepositoryImpl: dead probe.getClass() condition simplified to
  always load probe script from bootstrap CL (where it is defined)
- IndyDispatcher: added diagnostic logging for handler resolution failures

Test plan:
  ./gradlew :btrace-instr:test            — all instrumentor tests pass
  ./gradlew :btrace-instr:test -PupdateTestData — 382 dynamic golden files regenerated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents the develop-only branch model, build commands, module layout,
and PR checklist so automated review sessions always target the correct
integration branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jbachorik and others added 5 commits April 11, 2026 17:13
…ilure

IndyDispatcher lives in the bootstrap classloader. Adding SLF4J Logger
initialization to it caused LoggerFactory.getLogger() to run during
bootstrap class init, where no SLF4J provider is available. This made
Class.forName("...IndyDispatcher") throw ExceptionInInitializerError,
which HandlerRepositoryImpl's static initializer caught and swallowed —
leaving IndyDispatcher.repository null and all @OnMethod handlers as
permanent noops.

Handler resolution failures are already logged by HandlerRepositoryImpl
on the agent-classloader side. No logging is needed in the bootstrap
dispatcher itself.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dispatch

getBytecode(true) filtered to only BCP-required methods, which excluded
@OnMethod handlers (isBcpRequired() returns false when om != null).
The bootstrap-CL probe class therefore had no handler methods, causing
IndyDispatcher → HandlerRepositoryImpl.resolveHandler() →
publicLookup().findStatic() to throw NoSuchMethodException, which was
caught and stored as UNRESOLVED, permanently installing a noop
ConstantCallSite for every instrumented call site.

Fix: also include methods where getOnMethod() != null in the
bootstrap-CL class, and include their callees transitively.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…strap class

The INDY call site descriptor (generated by Instrumentor.invokeBTraceAction)
replaces AnyType with Object for JVM stack compatibility. The bootstrap-CL
probe class must have matching method descriptors so that
publicLookup().findStatic(probeClass, name, handlerType) succeeds.

Without this, handlers using @return AnyType or AnyType[] parameters
(oneliners with args, return-value captures, etc.) resolve as UNRESOLVED
and get a permanent noop CallSite, silencing those probe points.

Apply the same AnyType→Object descriptor substitution that copy() uses
for handler bytecode sent to defineHiddenClass (the old Java-15 path).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…r lookup

Two fixes:

1. BTraceProbeNode.getBytecode(true): also include @OnProbe handler methods
   (op != null) in the bootstrap-CL probe class. mapOnProbes() maps @OnProbe
   to synthetic @OnMethod entries which generate INDY call sites; the handler
   method bodies must be present in the bootstrap class for resolveHandler()
   to find them via findStatic().

2. BTraceRuntimeImpl_8.getCallerClassLoader/getCallerClass: skip
   org.openjdk.btrace.runtime.auxiliary.* frames when walking the call stack,
   mirroring BTraceRuntimeImpl_9's StackWalker-based skip of those frames.
   Before this fix, the probe handler frame (in bootstrap CL) was counted as
   the application caller, causing Class.forName inside BTraceUtils.field()
   and similar reflection utilities to use the bootstrap CL and fail with
   ClassNotFoundException for application classes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…trapClass

On JDK 8, BTraceRuntimeImpl_8.isBootstrapClass() calls
findBootstrapOrNullMtd.invoke() to check bootstrap classloader membership.
After 15 invocations the JVM inflates the reflective accessor, creating
sun/reflect/GeneratedMethodAccessorN via ClassLoader.defineClass().

This class-definition callback triggers BTraceTransformer.transform() which
calls BTraceClassWriter.getCommonSuperClass() → ClassInfo.inferClassLoader()
→ isBootstrapClass() → invoke() again. At call count > 15 the JVM tries to
inflate ANOTHER accessor (N+1), whose definition triggers another transform()
→ isBootstrapClass() → invoke() → accessor N+2, and so on indefinitely.

The resulting StackOverflowError propagates out of agent initialization,
causing testTraceAll (which instruments all classes via @OnMethod(clazz="/.*/"))
to fail on JDK 8 with "FATAL: Initialization failed: StackOverflowError".

Fix: add a ThreadLocal re-entrancy guard in isBootstrapClass(). While the
guard is set (i.e., we are already inside a bootstrap check that triggered
accessor inflation), any re-entrant call returns false conservatively.
This breaks the infinite recursion; the accessor class is defined once and
subsequent invoke() calls are served directly by the generated accessor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant