Skip to content

Add azmcp skill telemetry publish command for agent hook telemetry#1924

Draft
Copilot wants to merge 4 commits intomainfrom
copilot/add-skill-telemetry-command
Draft

Add azmcp skill telemetry publish command for agent hook telemetry#1924
Copilot wants to merge 4 commits intomainfrom
copilot/add-skill-telemetry-command

Conversation

Copy link
Contributor

Copilot AI commented Mar 3, 2026

Adds a new Azure.Mcp.Tools.Skills toolset exposing a single command for publishing skills-related telemetry events from agent hooks (VS Code, Claude, Copilot CLI, etc.) to be captured by the logging/monitoring infrastructure.

Command

azmcp skill telemetry publish \
  --events '[{"timestamp":"2026-03-03T23:11:41Z","event_type":"tool_invocation","tool_name":"azure_best_practices","session_id":"abc123"}]'

Accepts a required --events JSON array, validates it, logs each event via ILogger (feeds into Application Insights), and returns { "eventCount": N }.

Changes

  • New toolset tools/Azure.Mcp.Tools.Skills/ — follows standard toolset layout:
    • SkillsSetup registers command group hierarchy: skilltelemetrypublish
    • TelemetryPublishCommand — validates JSON, logs events, returns count
    • SkillsJsonContext — AOT-safe source-gen serialization context
    • 6 unit tests covering valid input, multiple events, empty array, invalid JSON, missing param
  • Program.cs — registers SkillsSetup
  • azmcp-commands.md — documents the new command with examples
Original prompt

This section details on the original issue you should resolve

<issue_title>[ONBOARD] Skill telemetry command</issue_title>
<issue_description>### Service / Tool Name

Azure.Skills.Telemetry

Contacts

skaluvak, kvenkatrajan

Intended Agent Scenarios

Ability to gather skills related telemetry from various clients like vscode, claude, copilot cli. This command is intended to be used from agent hooks to emit metrics to kusto tables.

Timeline

ASAP</issue_description>

<agent_instructions>Reference - https://github.com/microsoft/mcp/blob/ef8d0acfa3d468e4a4a3ffe957063f7bfb7fe366/servers/Azure.Mcp.Server/docs/new-command.md as needed.

For context, I want to publish skills related telemetry using agent hooks, which uses azure mcp cli/ exe. I want you to create a new command, that can be used from azmcp cli to publish a json array with data like this for example
{"timestamp":"2026-03-03T23:11:41.3587086Z","event_type":"tool_invocation","tool_name":"azure_best_practices","session_id":""}
{"timestamp":"2026-03-03T23:25:59.1918819Z","event_type":"skill_invocation","skill_name":"azure-prepare","session_id":""}

this data needs to be published to the respective kusto table. wherever the rest of the code is emitting its metrics. so that we can have aggregated information about number of skills invocated, number of mcp tools invoked from skills etc. </agent_instructions>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: saikoumudi <22682497+saikoumudi@users.noreply.github.com>
Copilot AI changed the title [WIP] Add skill telemetry command for Azure.Skills.Telemetry Add azmcp skill telemetry publish command for agent hook telemetry Mar 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Untriaged

Development

Successfully merging this pull request may close these issues.

[ONBOARD] Skill telemetry command

2 participants