Chat: OTel enrichment for agent context#318031
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR brings VS Code’s Copilot extension OpenTelemetry attributes into parity with the Copilot CLI runtime by introducing the canonical github.copilot.* namespace and dual-emitting it alongside existing legacy keys to preserve downstream compatibility (debug log, Chronicle, OTLP collectors, SQLite span store).
Changes:
- Add
GitHubCopilotAttrconstants (plus a SHA-256 helper) and dual-emit git/repo metadata undergithub.copilot.git.*. - Dual-emit the reasoning token metric under
gen_ai.usage.reasoning.output_tokenswhile retaining the legacy key. - Enrich spans with agent/tool/hook attributes, including structured
github.copilot.tool.parameters.*, plus docs and tests for the new telemetry shape.
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/util/node/crypto.ts | Adds a sync SHA-256 helper for hashing/redacting telemetry identifiers. |
| extensions/copilot/src/platform/otel/node/extractToolParameters.ts | Introduces structured extraction for tool parameter attributes (shell/file/skill/MCP). |
| extensions/copilot/src/platform/otel/node/test/extractToolParameters.spec.ts | Adds unit tests for structured tool parameter extraction. |
| extensions/copilot/src/platform/otel/common/workspaceOTelMetadata.ts | Dual-emits git/repo metadata into github.copilot.git.* and derives github.copilot.github.org. |
| extensions/copilot/src/platform/otel/common/test/workspaceOTelMetadata.spec.ts | Adds tests for dual-emit behavior and GitHub org derivation. |
| extensions/copilot/src/platform/otel/common/index.ts | Re-exports new constants/types for consumption across the extension. |
| extensions/copilot/src/platform/otel/common/genAiAttributes.ts | Defines the github.copilot.* attribute set and supporting types/constants. |
| extensions/copilot/src/extension/tools/vscode-node/toolsService.ts | Stamps structured github.copilot.tool.parameters.* onto execute_tool spans (gated where needed). |
| extensions/copilot/src/extension/prompt/node/chatMLFetcher.ts | Dual-emits reasoning token usage under the semantic-convention-aligned key. |
| extensions/copilot/src/extension/intents/node/toolCallingLoop.ts | Adds github.copilot.agent.type to invoke_agent spans. |
| extensions/copilot/src/extension/chat/vscode-node/chatHookService.ts | Enriches execute_hook spans with decision/duration/tool_names attributes. |
| extensions/copilot/docs/monitoring/sprint.plan.md | Documents locked design decisions and task breakdown for the telemetry parity sprint. |
| extensions/copilot/docs/monitoring/agent_monitoring.md | Documents namespaces/dual-emit policy and adds github.copilot.* rows (marking legacy keys). |
| .github/skills/otel/SKILL.md | Updates the OTel skill guidance with dual-emit rules and namespace guidance. |
Copilot's findings
- Files reviewed: 14/14 changed files
- Comments generated: 3
roblourens
previously approved these changes
May 22, 2026
…xtraction VS Code's ToolName enum uses snake_case identifiers (create_file, replace_string_in_file, apply_patch, insert_edit_into_file, multi_replace_string_in_file, edit_notebook_file, read_file). The initial CLI-derived FILE_TOOL_NAMES set only contained the Copilot CLI/Claude camelCase variants, so tool.parameters.edit_type and tool.parameters.file_path were never emitted on real VS Code tool spans. Add the missing names and extend classifyEditType to handle them. Verified end-to-end in Aspire on a trace where create_file now correctly emits edit_type=create with the file_path gated by captureContent.
- Move USAGE_REASONING_OUTPUT_TOKENS from GitHubCopilotAttr to GenAiAttr (its value is gen_ai.usage.reasoning.output_tokens, not a github.copilot.* key) - Accept Anthropic-style mcp__server__tool double-underscore format in addition to mcp_server_tool single-underscore - Add a test for the double-underscore variant
zhichli
changed the title
dmitrivMS
approved these changes
May 24, 2026
This was referenced May 27, 2026
dileepyavan
pushed a commit
that referenced
this pull request
May 27, 2026
Chat: OTel enrichment for agent context
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
What
Adds the canonical
github.copilot.*attribute namespace to Copilot Chat's OpenTelemetry output, alongside the existingcopilot_chat.*keys. Selected new signals:invoke_agentspans (github.copilot.git.{repository,branch,commit_sha},github.copilot.github.org).invoke_agent(github.copilot.agent.type=builtin|custom).execute_toolparameters for shell, file, skill, and MCP tools (github.copilot.tool.parameters.{command,file_path,edit_type,skill_name,mcp_server_name_hash,mcp_server_name,mcp_tool_name}).execute_hook(github.copilot.hook.{decision,duration,tool_names}).gen_ai.usage.reasoning.output_tokens.Content-sensitive attributes (raw commands, file paths, MCP server names) are gated on the existing
captureContentsetting. Identifiers that aren't gated are hashed (SHA-256) where appropriate.Why
Makes Copilot Chat's OTel output more useful for customer governance, audit, and dashboarding scenarios. The
gen_ai.*semantic conventions don't cover Copilot-specific signals (agent type, git context, structured tool parameters, hook outcomes), so a vendor namespace is the right home for them.Backwards compatibility
All existing
copilot_chat.*andgen_ai.usage.reasoning_tokensattributes continue to emit unchanged — the new keys are added in addition, not as a replacement. Downstream consumers (Agent Debug Log, Chronicle, OTLP exporters, the SQLite span store) keep working without modification. Legacy keys are marked inextensions/copilot/docs/monitoring/agent_monitoring.md.Tests
extractToolParameters.spec.ts— shell command truncation/gating; fileedit_typeclassification including VS Code's snake_case tool names; MCP single- and double-underscore name parsing;captureContentgating; unknown-tool fallback.workspaceOTelMetadata.spec.ts— dual-emit of git context;github.copilot.github.orgderivation; non-GitHub remote omitsorg.