feat: Add auto and wrapper instrumentation for @github/copilot-sdk#1932
Merged
Stephen Belanger (Qard) merged 5 commits intoMay 6, 2026
Merged
Conversation
Stephen Belanger (Qard)
force-pushed
the
feat/github-copilot-sdk-instrumentation
branch
from
30a27d5 to
9082765
Compare
Adds full Braintrust tracing support for the GitHub Copilot SDK
(`@github/copilot-sdk` v0.3+). Both auto-instrumentation (via
`--import braintrust/hook.mjs` / bundler plugins) and a manual
`wrapCopilotClient(client)` wrapper are provided.
Span tree produced per session:
Copilot Session (TASK) → Copilot Turn (TASK) → github.copilot.llm (LLM)
→ tool: <name> (TOOL)
→ Agent: <name> (TASK, sub-agents)
Token metrics on LLM spans use the Anthropic cache-token helpers:
prompt_tokens, completion_tokens, prompt_cached_tokens,
prompt_cache_creation_tokens, completion_reasoning_tokens, reasoning_tokens,
tokens. Copilot-specific billing data (cost multiplier, quota snapshots,
copilot_usage) flows into namespaced metadata rather than metrics.
Also fixes imports.test.ts to skip node_modules and .d.ts files when
scanning for dynamic import violations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n to 3.10.0
- Remove unused `ChannelMessage` import in github-copilot-plugin.ts (lint error)
- Fix `anthropic.test.ts`: system prompt says "Just the poem" so asserting
`output.contains("shakespeare")` is wrong — the model correctly returns only
the poem content without mentioning the author
- Apply all pending changesets via `changeset version` to bump braintrust to
3.10.0, so the API compatibility test detects a `minor` version bump rather
than `none` and does not flag optional interface additions as breaking
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t and formatting - Remove gitHubCopilot from InstrumentationConfig and BraintrustPluginConfig inline types (api-compat treats any change to the inline type string as a modification; use getIntegrationConfig dynamic lookup instead) - Fix stale Shakespeare assertions in anthropic.test.ts (rebase left the old test content while main had updated it to assert "btanthropicstreamok") - Fix CHANGELOG.md formatting (missing blank line before section heading) - Add vscode-jsonrpc as explicit e2e dep so pnpm resolves the transitive dep of @github/copilot-sdk in the isolated scenario install Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stephen Belanger (Qard)
force-pushed
the
feat/github-copilot-sdk-instrumentation
branch
from
32a1497 to
4c433b6
Compare
…jsonrpc vscode-jsonrpc is a direct dep of @github/copilot-sdk that pnpm doesn't hoist into the scenario's isolated install; adding it explicitly requires regenerating the scenario's pnpm-lock.yaml so the hermetic e2e can install with --frozen-lockfile. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Abhijeet Prasad (AbhiPrasad)
approved these changes
May 6, 2026
Stephen Belanger (Qard)
deleted the
feat/github-copilot-sdk-instrumentation
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--import braintrust/hook.mjsand bundler plugins) and awrapCopilotClient(client)manual wrapper for@github/copilot-sdkv0.3+prompt_tokens,completion_tokens,prompt_cached_tokens,prompt_cache_creation_tokens,completion_reasoning_tokens,reasoning_tokens,tokenscost,quotaSnapshots,copilotUsage) flow into namespacedgithub_copilot.*metadata, not metricsCopilotClient.{createSession,resumeSession}indist/client.jsandCopilotSession.sendAndWaitindist/session.js(both ESM and CJS)imports.test.tsto skipnode_modulesand.d.tsfiles during the dynamic-import scan (those directories can be populated by test fixture installs)Architecture notes
The Copilot SDK is an agent-runtime client (JSON-RPC over a CLI subprocess), not a chat-completions wrapper — so instrumentation follows the lifecycle/processor pattern used for
@anthropic-ai/claude-agent-sdkrather than the per-method pattern used for Groq/OpenAI.Span lifecycle is driven by the
assistant.turn_start/turn_endevent stream andassistant.usage(per-LLM-call metrics). Tool and sub-agent spans usetool.execution_start/completeandsubagent.started/completed/failedevents. Session close is triggered by the injectedonSessionEndhook.Parent span IDs are resolved by awaiting
span.export()(same pattern asclaude-agent-sdk-plugin.ts). Async event handlers are chained throughstate.processing: Promise<void>to preserve ordering.Test plan
tsc --noEmitclean (no type errors)pnpm run buildsucceedspnpm run lint— warnings only (same pattern as rest of codebase)pnpm run formattingcleanextractMetricsFromUsage(all token/cache/reasoning/billing fields), plugin lifecycle, orchestrion config shape,BraintrustPluginwiringe2e/scenarios/github-copilot-instrumentation/requires a GitHub Copilot subscription or a BYOK provider endpoint (BRAINTRUST_E2E_MODEL_BASE_URL)🤖 Generated with Claude Code