Add BTrace extensions and MCP server: runtime contracts, AI/ML observability, LLM-driven diagnostics by jbachorik · Pull Request #810 · btraceio/btrace

jbachorik · 2026-03-28T15:54:02Z

Summary

Adds five new modules that extend BTrace with runtime contract enforcement, AI/ML observability, and AI-agent integration:

btrace-contracts — General-purpose runtime behavioral contracts: latency budgets, call-rate limits (sliding-window), range checks, null-safety, boolean assertions, and tagged code path profiling. User-supplied tags let you compare any two implementations side by side (e.g. "cached" vs "direct", "v1" vs "v2").
btrace-mcp-server — Model Context Protocol server exposing BTrace as 7 MCP tools (list_jvms, deploy_oneliner, deploy_script, list_probes, send_event, detach_probe, exit_probe) plus 3 diagnostic prompt templates. Enables LLMs like Claude to attach to JVMs and run BTrace probes via natural language.
btrace-llm-trace — LLM inference observability: token counts (input/output/cache-read/cache-creation), latency (min/avg/max), cost estimation with cache savings, streaming TTFT. Covers Langchain4j, Spring AI, OpenAI Java SDK, Anthropic Java SDK patterns.
btrace-rag-quality — RAG pipeline observability: vector DB query latency, similarity score tracking (top/low/spread), empty retrieval rates, chunk token budgets, end-to-end pipeline timing (retrieval vs generation). Targets Pinecone, Milvus, Weaviate, Chroma, pgvector, Qdrant.
btrace-gpu-bridge — GPU compute and model inference tracing: ONNX Runtime, DJL, TensorFlow Java session tracking, batch throughput, device memory allocation/peak tracking, native FFM call profiling (cuBLAS, cuDNN).

Build system upgrade: Gradle 8 → 9.5.1

This PR also upgrades the build toolchain to Gradle 9.5.1. The upgrade was required to unblock CI — the previous combination of Gradle 8 + Spotless 7 + google-java-format stopped working on JDK 26 runners because getDiagnostics() changed its return type from Queue to List between JDK versions, and the old formatter bound to the old type at the bytecode level.

Changes made:

Component	Old	New	Reason
Gradle	8.14.4	9.5.1	Spotless 8.x requires Gradle 9; unblocks JDK 26 runners
Spotless	7.x	8.5.1	First version compiled for JDK 26; fixes `getDiagnostics()` incompatibility
Shadow plugin	`com.github.johnrengelman.shadow` 8.1.1	`com.gradleup.shadow` 9.4.1	Original plugin abandoned; maintained fork required for Gradle 9
ospackage	11.11.2	12.3.0	API compatibility with Gradle 9
Build JDK	11	21	Gradle 9.5.1 requires JDK 17+ to run the build daemon

Gradle 9 API removals fixed:

fileMode = 0644 → filePermissions { unix(0644) } (btrace-dist)
project.docsDir → layout.buildDirectory.dir(...) (btrace-dist)
junit-platform-launcher added to testRuntimeOnly in common.gradle (Gradle 9 no longer auto-provides it)

Design principles

All extensions follow the same architecture:

Zero allocation on hot paths — fluent builders (llm.call("model").inputTokens(1500).record()) use ThreadLocal pooling, reusing one instance per thread instead of allocating per call
Lock-free concurrency — all statistics use AtomicLong with CAS loops for min/max, no locks anywhere
Zero external dependencies — instrument existing client library classes, no additional JARs required
Standard BTrace extension pattern — @ServiceDescriptor interfaces with @Nullable on non-primitive return types, implementations extend Extension, auto-discovered via the Gradle extension plugin

Rebase notes

This branch has been rebased onto develop and updated to match the current codebase:

Migrated all packages from org.openjdk.btrace.* → io.btrace.* (consistent with the develop migration)
Moved source files from src/api/java/ + src/impl/java/ to src/main/java/ (the only source set compiled by the BTrace extension Gradle plugin)
Added @Nullable on all non-primitive return types in service interfaces (required by the validateServiceApis task)
Added org.jetbrains:annotations:26.1.0 as a compile-only dependency

Files

Area	Files	Lines
Runtime contracts extension	3 source + 1 test	~700
MCP server	12 source files	~2,000
LLM trace extension	5 source + 1 test	~1,100
RAG quality extension	5 source + 1 test	~800
GPU bridge extension	5 source + 1 test	~800
Build system upgrade	`common.gradle`, `settings.gradle`, `continuous.yml`, `btrace-dist/build.gradle`	~50
Total	~40 files	~5,450

Test plan

Unit tests pass for btrace-contracts
Unit tests pass for btrace-llm-trace
Unit tests pass for btrace-rag-quality
Unit tests pass for btrace-gpu-bridge
btrace-mcp-server compiles cleanly
Gradle 9.5.1 build passes locally
Sample BTrace scripts compile against extension APIs
Existing BTrace tests unaffected

🤖 Generated with Claude Code

This change is

Implements an MCP (Model Context Protocol) server that exposes BTrace operations as tools over stdio JSON-RPC transport. This allows LLM clients (Claude Desktop, Claude Code, Cursor, etc.) to attach to running JVMs and deploy diagnostic probes conversationally. MCP tools: list_jvms, deploy_oneliner, deploy_script, list_probes, send_event, detach_probe, exit_probe. MCP prompts: diagnose_slow_endpoint, find_exception_source, profile_method. Also makes Client.connectAndListProbes, isDisconnected, and disconnect public for cross-module access. https://claude.ai/code/session_012KcpiFxvscxzWWgN5LiRcB

Use Java 11 source/target (MCP server needs 11+ APIs) while keeping the project's standard JDK 11 toolchain from common.gradle. Restore tools.jar compileOnly dependency for sun.jvmstat access. https://claude.ai/code/session_012KcpiFxvscxzWWgN5LiRcB

Documents MCP tools, prompts, build instructions, and configuration for Claude Desktop, Claude Code, and Cursor integration. https://claude.ai/code/session_012KcpiFxvscxzWWgN5LiRcB

New BTrace extension that provides an LlmTraceService for recording and aggregating LLM API call metrics from BTrace scripts: - Per-model token counts (input/output), latency (min/avg/max) - Streaming call tracking with time-to-first-token - Tool/function call counting - Error tracking by type - Built-in cost estimation for Claude, GPT, Gemini model families - Thread-safe lock-free implementation using AtomicLong counters Extension structure follows btrace-metrics/btrace-utils pattern: - API: LlmTraceService interface with @ServiceDescriptor - Impl: LlmTraceServiceImpl extending Extension, zero external deps - Tests: 14 tests covering aggregation, cost estimation, concurrency Includes sample BTrace script (LlmTrace.java) demonstrating usage with Langchain4j ChatLanguageModel instrumentation. https://claude.ai/code/session_012KcpiFxvscxzWWgN5LiRcB

The fluent builder (llm.call("model").inputTokens(...).record()) now reuses a per-thread CallRecordImpl instance instead of allocating a new object on every call. This eliminates GC pressure on hot-path instrumentation while preserving the ergonomic fluent API. Also includes earlier API refinements: simplified recording methods, cache token tracking, embedding support, and comprehensive tests. https://claude.ai/code/session_012KcpiFxvscxzWWgN5LiRcB

…nsions Three new AI/computing extensions following the BTrace extension pattern: - btrace-rag-quality: RAG pipeline observability — vector DB query latency, similarity scores, empty retrieval rates, chunk quality. Supports Pinecone, Milvus, Weaviate, Chroma, pgvector, Qdrant. - btrace-vibe-guard: Runtime behavioral contracts for AI-generated code — latency budgets, call rate limits, range checks, null-safety enforcement, AI vs human code path comparison. - btrace-gpu-bridge: GPU compute and model inference tracing — ONNX Runtime, DJL, TensorFlow Java. Tracks inference latency, batch throughput, device memory allocation, native FFM calls to CUDA/cuBLAS. All three use ThreadLocal-pooled fluent builders (zero allocation), lock-free AtomicLong statistics, and include comprehensive tests. Sample BTrace scripts included for each. https://claude.ai/code/session_012KcpiFxvscxzWWgN5LiRcB

…ce.* - Move source files from src/api/java + src/impl/java to src/main/java (BTrace extension plugin only compiles sourceSets.main) - Update package declarations and imports in all 5 modules: btrace-mcp-server, btrace-llm-trace, btrace-rag-quality, btrace-vibe-guard, btrace-gpu-bridge - Fix build.gradle: update plugin IDs, service class names, Main-Class - Add @nullable on all non-primitive service interface return types (required by validateServiceApis task for shim compatibility) - Add org.jetbrains:annotations:26.1.0 as compile-only dependency - Fix VibeGuardServiceImpl: use "AI Code Paths" header when no human paths recorded, avoiding spurious "Human" in aiOnlyTracking test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix pre-existing Google Java Format violations introduced by develop-branch commits that modified imports and call-sites without re-running spotlessApply. All changes are mechanical: import reordering and line-length wrapping only. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Adds five new modules to bring AI/ML observability and LLM-driven control to BTrace: an MCP server exposing BTrace operations as tools/prompts for LLMs, plus four extension modules for LLM inference tracing, RAG pipeline quality, runtime contracts for AI-generated code, and GPU/inference tracing. The changes plug into the existing extension Gradle plugin pipeline and are wired in via settings.gradle.

Changes:

New btrace-mcp-server module implementing 7 MCP tools (list_jvms, deploy_oneliner, deploy_script, list_probes, send_event, detach_probe, exit_probe) plus diagnostic prompt templates over the existing Client API.
Four new BTrace extensions (btrace-llm-trace, btrace-rag-quality, btrace-vibe-guard, btrace-gpu-bridge) following the standard @ServiceDescriptor + Extension pattern with lock-free statistics, plus per-module unit tests.
Sample BTrace scripts under btrace-dist/src/main/resources/samples/ demonstrating each new extension, and minor spotless-driven reformatting touch-ups across btrace-compiler, btrace-agent, btrace-client.

Reviewed changes

Copilot reviewed 47 out of 47 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`settings.gradle`	Includes the four new extension subprojects in the build.
`btrace-mcp-server/README.md`	New MCP server docs; still references old `org.openjdk.btrace.mcp.BTraceMcpServer` class name.
`btrace-mcp-server/src/main/java/io/btrace/mcp/BTraceMcpServer.java`	MCP server entry point.
`btrace-mcp-server/src/main/java/io/btrace/mcp/ClientManager.java`	Caches `Client` instances per pid/port; registration side is currently unused by deploy handlers.
`btrace-mcp-server/src/main/java/io/btrace/mcp/tools/ListJvmsHandler.java`	Lists attachable JVMs via jvmstat + attach API; heavy per-PID probing.
`btrace-mcp-server/src/main/java/io/btrace/mcp/tools/DeployOnelinerHandler.java`	Submits a one-liner; missing `ClientManager.registerClient` after submit.
`btrace-mcp-server/src/main/java/io/btrace/mcp/tools/DeployScriptHandler.java`	Submits a script; same missing registration as one-liner handler.
`btrace-mcp-server/src/main/java/io/btrace/mcp/tools/ListProbesHandler.java`	Wraps `Client.connectAndListProbes` with a latch + timeout.
`btrace-mcp-server/src/main/java/io/btrace/mcp/tools/SendEventHandler.java`	Sends a (possibly named) event via an existing session.
`btrace-mcp-server/src/main/java/io/btrace/mcp/tools/DetachProbeHandler.java`	Detaches a probe via an existing session.
`btrace-mcp-server/src/main/java/io/btrace/mcp/tools/ExitProbeHandler.java`	Stops/removes a probe via an existing session.
`btrace-mcp-server/src/main/java/io/btrace/mcp/prompts/*.java`	Diagnostic prompt templates exposed via MCP.
`btrace-extensions/btrace-llm-trace/**`	LLM inference observability service + extension + tests.
`btrace-extensions/btrace-rag-quality/**`	RAG pipeline quality service + extension + tests.
`btrace-extensions/btrace-vibe-guard/**`	Runtime contracts service + extension + tests.
`btrace-extensions/btrace-gpu-bridge/**`	GPU/inference tracing service + extension + tests.
`btrace-dist/src/main/resources/samples/LlmTrace.java`	Sample using stale package and non-existent/wrong-arity service methods.
`btrace-dist/src/main/resources/samples/RagQuality.java`	Sample using stale `org.openjdk.btrace.*` packages.
`btrace-dist/src/main/resources/samples/VibeGuard.java`	Sample using stale `org.openjdk.btrace.*` packages.
`btrace-dist/src/main/resources/samples/GpuBridge.java`	Sample using stale `org.openjdk.btrace.*` packages.
`btrace-compiler/`, `btrace-agent/`, `btrace-client/**`	Minor spotless/import-order reformatting only.

@injected

…ion lookup Samples (GpuBridge, LlmTrace, RagQuality, VibeGuard): - Migrate from org.openjdk.btrace.* to io.btrace.* packages - Use import static io.btrace.core.BTraceUtils.* for println/strcat - Use explicit io.btrace.core.annotations.* imports - Fix LlmTrace API usage to match actual LlmTraceService interface VerifierVisitor.isDeclaredExtensionService: - Add primary check via annotation processing Elements API (sees -cp JARs) by looking for @ServiceDescriptor annotation on the type element - Keep jar-manifest scanning as fallback for JVM-classpath-visible jars - This allows btracec -cp ext-api.jar to validate @injected extension fields Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@SuppressWarnings

- McpProtocol.java:268/270 — wrap Double.parseDouble and Long.parseLong in try/catch, rethrow as IllegalArgumentException with position context - BTraceMcpServer.java:116 — remove unused params parameter from handleInitialize - README.md:44/62/78/96 — fix class name from org.openjdk.btrace.mcp to io.btrace.mcp.BTraceMcpServer - GpuBridgeServiceImpl.java:207 — remove unused totalNative accumulator variable - ListJvmsHandler.java:26-28,55 — replace sun.jvmstat.monitor.* with VirtualMachine.list(); add @SuppressWarnings on unused arguments param - DeployOnelinerHandler.java:160 — register client with ClientManager after successful submission - DeployScriptHandler.java:160 — register client with ClientManager after successful submission - LlmTrace.java:69 — remove unused @ProbeMethodName String methodName parameter Also apply spotless import ordering to all new btrace-mcp-server source files. Co-Authored-By: muse <muse@noreply>

jbachorik · 2026-05-17T12:39:04Z

@github-code-quality review

jbachorik · 2026-05-17T12:39:13Z

@copilot review

Copilot · 2026-05-17T12:42:31Z

@copilot review

Reviewed the current branch after 77c1144 and revalidated it with :btrace-dist:build plus the four new extension test tasks; everything is passing and I didn’t find any additional issues to fix.

Replace the narrowly-named `btrace-vibe-guard` / `VibeGuardService` with the general-purpose `btrace-contracts` / `ContractService`. The extension has nothing AI-specific about it: it enforces runtime behavioral contracts (latency budgets, call-rate limits, null checks, range assertions) and profiles tagged code paths for any user-supplied labels. Changes: - Rename module, package, and service classes throughout - Replace `trackAiCodePath`/`trackHumanCodePath` with `trackCodePath(contract, durationNanos, String tag)` — callers supply any label; getSummary() renders all tags side-by-side and shows faster/slower when exactly two tags exist - Move from Lesson 11 (AI/LLM Observability) to its own Lesson 11; renumber old Lesson 11→12 and Lesson 12→13 - Rename sample VibeGuard.java → ContractCheck.java - Update settings.gradle, docs (tutorial, README), and PR description Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Captures the design for a portable Technology Compatibility Kit covering all 11 axes of the BTrace extension contract: structural, lifecycle, behavioral, and performance compliance. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ospackage 12.3.0 Gradle 9.5.1 supports running on JDK 17–26. Spotless 8.5.1 bundles google-java-format 1.28.0 which uses reflection to call getDiagnostics(), avoiding the JDK 26 incompatibility (return type changed Queue→List). Migration fixes: - Replace fileMode with filePermissions { unix(...) } (Gradle 9 API removal) - Replace project.docsDir with layout.buildDirectory.dir(...) (Gradle 9 removal) - Switch shadow plugin ID from com.github.johnrengelman.shadow (abandoned) to com.gradleup.shadow 9.4.1 (maintained fork, same runtime package names) - Bump ospackage from 11.11.2 to 12.3.0 (Gradle 9 API compatibility) - Bump CI build JDK from 11 to 21 (Gradle 9 requires JDK 17+ to run) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Reformatted with google-java-format 1.28.0 (via Spotless 8.5.1) to resolve spotlessJavaCheck failures on the CI build. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Resolve conflicts between Gradle 9.5.1 upgrade and develop: - continuous.yml: keep Java 21 build JDK (Gradle 9.5.1 requires JDK 17+; develop's Java 24+11 combo is incompatible with Gradle 9) - btrace-dist/build.gradle: use develop's layout.buildDirectory.dir() API with our filePermissions { unix(0644) } fix for Gradle 9 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Plan 1: btrace-ext-validator module + btrace-tck structural suite Plan 2: lifecycle suite (load/init/inject/close) + behavioral suite (null safety, shim fallback, concurrency, required service) Plan 3: performance suite (baseline/overhead/budget) + btrace-tck-gradle-plugin Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Strips all extension-registry code that was added by a previous agent session but is unrelated to the AI/LLM extensions and Gradle 9 migration that PR #810 covers. Removed: BTraceRegistryConfig, generateRegistryEntry/validateRegistryEntry /verifyRegistryCoordinates/updateRegistryCatalog tasks, RegistryExtensionSource, Jackson deps from btrace-client and btrace-gradle-plugin, update-extension-registry CI job, and registry documentation sections. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Gradle 9 no longer auto-provides the JUnit Platform launcher on the test classpath. Add it globally in common.gradle so all subprojects can run JUnit 5 tests. Fixes :btrace-boot:test FAILED with "Failed to load JUnit Platform". Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…urce set btrace-agent has a java24 source set requiring languageVersion=24. CI was failing because the build job only installed JDK 21. Add a JDK 24 setup step (same pattern as develop) so the toolchain resolver finds it; JDK 21 remains the last setup and therefore the default JAVA_HOME for Gradle 9.5.1. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…rces Main.java, Installer.java, and ExtensionLister.java all imported io.btrace.registry.* and called RegistrySupport — classes that live in shared/registry-client which is not part of this branch. Remove all registry-dependent code paths and their test cases. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik changed the title ~~Add btrace-mcp-server module for LLM-driven JVM diagnostics~~ Add AI/ML extensions and MCP server for LLM-driven JVM diagnostics Mar 28, 2026

claude and others added 7 commits May 16, 2026 22:54

Add README for btrace-mcp-server module

aea57e4

Documents MCP tools, prompts, build instructions, and configuration for Claude Desktop, Claude Code, and Cursor integration. https://claude.ai/code/session_012KcpiFxvscxzWWgN5LiRcB

jbachorik force-pushed the claude/btrace-ai-extensions-920Ie branch from b10cc79 to 21359cb Compare May 16, 2026 21:32

github-code-quality Bot found potential problems May 16, 2026

View reviewed changes

jbachorik requested a review from Copilot May 17, 2026 07:32

Copilot started reviewing on behalf of jbachorik May 17, 2026 07:32 View session

Copilot AI reviewed May 17, 2026

View reviewed changes

github-code-quality Bot found potential problems May 17, 2026

View reviewed changes

Comment thread btrace-dist/src/main/resources/samples/LlmTrace.java Fixed

jbachorik and others added 2 commits May 17, 2026 11:40

Merge branch 'develop' into claude/btrace-ai-extensions-920Ie

a41e0c3

github-code-quality Bot found potential problems May 17, 2026

View reviewed changes

Comment thread btrace-mcp-server/src/main/java/io/btrace/mcp/tools/ListJvmsHandler.java Fixed

Copilot started work on behalf of jbachorik May 17, 2026 12:39 View session

Copilot finished work on behalf of jbachorik May 17, 2026 12:43

jbachorik changed the title ~~Add AI/ML extensions and MCP server for LLM-driven JVM diagnostics~~ Add BTrace extensions and MCP server: runtime contracts, AI/ML observability, LLM-driven diagnostics May 17, 2026

jbachorik and others added 5 commits May 17, 2026 16:29

style: apply google-java-format to btrace-compiler sources

2be7427

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs(internal): add extension TCK design spec

02da1ce

Captures the design for a portable Technology Compatibility Kit covering all 11 axes of the BTrace extension contract: structural, lifecycle, behavioral, and performance compliance. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

style: apply google-java-format to btrace-mcp-server and other modules

cd4a35e

Reformatted with google-java-format 1.28.0 (via Spotless 8.5.1) to resolve spotlessJavaCheck failures on the CI build. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

github-code-quality Bot found potential problems May 17, 2026

View reviewed changes

Comment thread btrace-mcp-server/src/main/java/io/btrace/mcp/tools/ListJvmsHandler.java Dismissed

jbachorik and others added 7 commits May 17, 2026 18:02

docs: add MCP server user guide

6fc2c35

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik merged commit 4eb688d into develop May 17, 2026
13 checks passed

jbachorik deleted the claude/btrace-ai-extensions-920Ie branch May 17, 2026 16:58

Conversation

jbachorik commented Mar 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Build system upgrade: Gradle 8 → 9.5.1

Design principles

Rebase notes

Files

Test plan

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jbachorik commented May 17, 2026

Uh oh!

Uh oh!

jbachorik commented May 17, 2026

Uh oh!

Copilot AI commented May 17, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jbachorik commented Mar 28, 2026 •

edited

Loading