Skip to content

Chat: OTel enrichment for Claude agent sessions#318472

Merged
zhichli merged 2 commits into
mainfrom
zhichli/otel-claude
May 27, 2026
Merged

Chat: OTel enrichment for Claude agent sessions#318472
zhichli merged 2 commits into
mainfrom
zhichli/otel-claude

Conversation

@zhichli
Copy link
Copy Markdown
Member

@zhichli zhichli commented May 27, 2026

Overview

Follow-up to #318031, which introduced the canonical github.copilot.* OTel attribute namespace on the foreground Copilot Chat agent. This PR extends the same enrichment to Claude Code agent sessions so traces carry parity attributes across agent surfaces.

What

  • invoke_agent claude (claudeOTelTracker) — now emits:
    • github.copilot.agent.type: 'builtin'
    • github.copilot.git.{repository,branch,commit_sha} and github.copilot.github.org (via workspaceMetadataToOTelAttributes(resolveWorkspaceOTelMetadata(gitService)), mirroring toolCallingLoop).
  • execute_tool for Claude tool calls (claudeMessageDispatch.handleAssistantMessage) — now uses an injected tool-parameter extractor (node-layer provided) to emit:
    • Ungated: github.copilot.tool.parameters.edit_type, skill_name, mcp_server_name_hash, mcp_tool_name.
    • Content-gated (captureContent): command, file_path, mcp_server_name.
  • execute_hook (claudeMessageDispatch.handleHookResponse) — now emits:
    • github.copilot.hook.decisionpass (outcome success) / block (exit code 2) / non_blocking_error (other failure).
    • github.copilot.hook.duration — seconds, measured between hookStarted and hookResponse via a new hookStartTimes map on MessageHandlerState.
  • extractToolParameters / genAiAttributes — added Claude tool-name aliases so the classifier recognises Bash, Read, Edit, MultiEdit, Write, NotebookEdit.
  • Truncation fixgen_ai.tool.call.arguments now truncates with otelService.config.maxAttributeSizeChars to avoid oversized OTel attributes being dropped.

Note: github.copilot.hook.tool_names is intentionally not emitted for Claude — the Claude SDK's SDKHookStartedMessage does not surface a tool_name to the extension. The user-facing doc already qualifies this attribute as "When tool-scoped".

Why

After #318031, dashboards built on the canonical github.copilot.* namespace were missing rows for the Claude agent path — its invoke_agent / execute_tool / execute_hook spans were only partially queryable. This brings Claude to parity with the built-in agent's enrichment while preserving extension layering constraints.

Backwards compatibility

All existing attributes (gen_ai.*, copilot_chat.*) remain unchanged. Only additive attribute writes; no behavioural change to dispatch or hook handling.

Docs

Updated extensions/copilot/docs/monitoring/:

  • agent_monitoring.md — Claude invoke_agent claude attribute table now lists the new github.copilot.{agent.type,git.*,github.org} keys; added a proper attribute table for Claude execute_tool listing tool.parameters.*.
  • agent_monitoring_arch.md — corrected the github.copilot.* namespace description to reflect that it's the canonical Copilot namespace (extension-emitted enrichment + CLI SDK internal metrics), not just SDK-internal.

Tests

  • extractToolParameters.spec.ts — adds Claude-name classification (Bash, Read, Edit, MultiEdit, Write, NotebookEdit).
  • claudeMessageDispatch.spec.ts — adds hook.decision / hook.duration cases and tool.parameters.* emission.
  • claudeCodeAgentOTel.spec.ts — exercised via existing end-to-end coverage.
  • All 89 tests pass across the three affected spec files; typecheck clean.

Copilot AI review requested due to automatic review settings May 27, 2026 00:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the Copilot extension’s canonical github.copilot.* OpenTelemetry enrichment to the Claude agent session flow so Claude invoke_agent / execute_tool / execute_hook spans become queryable on the same dimensions as built-in Copilot Chat spans.

Changes:

  • Enriches Claude invoke_agent spans with github.copilot.agent.type and workspace git/GitHub org attributes via workspace metadata resolution.
  • Emits structured github.copilot.tool.parameters.* attributes for Claude tool calls using extractToolParameters, with content-sensitive fields gated behind captureContent.
  • Records hook completion attributes (github.copilot.hook.decision, github.copilot.hook.duration) and extends tool-name classification to Claude’s capitalized tool names.
Show a summary per file
File Description
extensions/copilot/src/platform/otel/node/test/extractToolParameters.spec.ts Adds coverage for Claude-capitalized tool-name classification.
extensions/copilot/src/platform/otel/node/extractToolParameters.ts Extends edit-type classification to Claude tool names.
extensions/copilot/src/platform/otel/common/genAiAttributes.ts Adds Claude tool names to shell/file tool-name sets used by OTel extraction.
extensions/copilot/src/extension/chatSessions/claude/node/claudeOTelTracker.ts Adds canonical Copilot git/org/agent-type attributes to Claude root spans.
extensions/copilot/src/extension/chatSessions/claude/node/claudeCodeAgent.ts Wires additional per-request timing state for hook duration tracking.
extensions/copilot/src/extension/chatSessions/claude/common/test/claudeMessageDispatch.spec.ts Adds tests for tool parameter emission and hook decision/duration attributes.
extensions/copilot/src/extension/chatSessions/claude/common/claudeMessageDispatch.ts Emits tool parameter attributes and hook decision/duration attributes on spans.

Copilot's findings

  • Files reviewed: 9/9 changed files
  • Comments generated: 1

DonJayamanne
DonJayamanne previously approved these changes May 27, 2026
@zhichli
Copy link
Copy Markdown
Member Author

zhichli commented May 27, 2026

Pushed 55224ab to fix CI:

  • Layering violation (root cause of the 3 CI failures): common/claudeMessageDispatch.ts was importing extractToolParameters from platform/otel/node/, which violates the common→node layering rule. Fixed by injecting the function through MessageHandlerState from the node-layer caller (claudeCodeAgent.ts); the test file uses a small local stub. Verified with eslint on the affected directories.
  • Inline review feedback: addressed @copilot-pull-request-reviewer's commenttruncateForOTel(...) for gen_ai.tool.call.arguments now passes maxAttributeSizeChars.

89 tests pass; typecheck clean (the unrelated AgentTask / @vscode/copilot-api errors exist on main and are not from this PR).

@zhichli zhichli merged commit 178fb65 into main May 27, 2026
26 checks passed
@zhichli zhichli deleted the zhichli/otel-claude branch May 27, 2026 05:54
@vs-code-engineering vs-code-engineering Bot added this to the 1.123.0 milestone May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants