Skip to content

Fixes invalid telemetry tag for ToolArea#346

Merged
conniey merged 14 commits intomicrosoft:mainfrom
conniey:toolarea-fixes
Sep 8, 2025
Merged

Fixes invalid telemetry tag for ToolArea#346
conniey merged 14 commits intomicrosoft:mainfrom
conniey:toolarea-fixes

Conversation

@conniey
Copy link
Copy Markdown
Member

@conniey conniey commented Sep 5, 2025

What does this PR do?

  • Fixes telemetry issue where ToolArea tag would be filled in with ToolName if area could not be found. Sets result to null so misleading information is not sent.
    • Uses direct mapping of tool name to IAreaSetup to avoid any errors from trying to parse the full name into its components and inferring the tool's area.
  • Add tests.

GitHub issue number?

Fixees: https://github.com/Azure/azure-mcp-pr/issues/355

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Updated servers/Azure.Mcp.Server/CHANGELOG.md and/or servers/Fabric.Mcp.Server/CHANGELOG.md for product changes (features, bug fixes, UI/UX, updated dependencies)
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Updated command list in /docs/azmcp-commands.md and/or /docs/fabric-commands.md
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
  • Extra steps for Azure MCP Server tool changes:
    • Updated test prompts in /docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

Copilot AI review requested due to automatic review settings September 5, 2025 11:10
@conniey conniey requested a review from a team as a code owner September 5, 2025 11:10
@github-project-automation github-project-automation Bot moved this to Untriaged in Azure MCP Server Sep 5, 2025
@conniey conniey changed the title Fixes Telemetry tag for ToolArea Fixes invalid telemetry tag for ToolArea Sep 5, 2025
Copilot AI reviewed Sep 5, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a telemetry issue where the ToolArea tag would incorrectly fall back to using the tool name when the area could not be found, resulting in misleading telemetry data. The fix implements a direct mapping approach from tool names to area setups and returns null when the area cannot be determined.

  • Updated the CommandFactory to maintain a direct mapping from command names to their corresponding IAreaSetup objects
  • Modified the telemetry logic to avoid fallback to tool name when area lookup fails
  • Added comprehensive test coverage for the new functionality

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
CommandFactory.cs Implements direct mapping from command names to areas and refactors area lookup logic
CommandFactoryToolLoader.cs Removes fallback logic that used tool name when area lookup failed
CommandFactoryTests.cs Adds extensive test coverage for area lookup and command dictionary creation
McpRuntimeTests.cs Adds test to verify activity tags are properly set during tool execution
CommandTestsBase.cs Adds overloaded method to support testing with different MCP clients
CommandTests.cs Adds integration test to verify tool functionality across different server modes

Comment thread core/Azure.Mcp.Core/tests/Azure.Mcp.Core.UnitTests/Commands/CommandFactoryTests.cs Outdated
Comment thread core/Azure.Mcp.Core/tests/Azure.Mcp.Core.LiveTests/CommandTests.cs Outdated
Comment thread core/Azure.Mcp.Core/src/Commands/CommandFactory.cs Outdated
Comment thread core/Azure.Mcp.Core/tests/Azure.Mcp.Core.UnitTests/Commands/CommandFactoryTests.cs Outdated
Comment thread core/Azure.Mcp.Core/tests/Azure.Mcp.Core.UnitTests/Commands/CommandFactoryTests.cs Outdated
@github-project-automation github-project-automation Bot moved this from Untriaged to In Progress in Azure MCP Server Sep 8, 2025
@joshfree
Copy link
Copy Markdown
Member

joshfree commented Sep 8, 2025

https://dev.azure.com/azure-sdk/internal/_build/results?buildId=5308623&view=ms.vss-test-web.build-test-results-tab&runId=56723254&resultId=100042&paneView=debug
image

joshfree
joshfree reviewed Sep 8, 2025
View reviewed changes
Comment thread core/Azure.Mcp.Core/tests/Azure.Mcp.Core.LiveTests/CommandTests.cs Outdated
@joshfree joshfree added this to the 2025-09 milestone Sep 8, 2025
@conniey conniey enabled auto-merge (squash) September 8, 2025 20:43
@conniey conniey merged commit f9295a1 into microsoft:main Sep 8, 2025
26 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Azure MCP Server Sep 8, 2025
@conniey conniey deleted the toolarea-fixes branch September 9, 2025 00:11
colbytimm pushed a commit to colbytimm/microsoft-mcp that referenced this pull request Sep 27, 2025
@conniey @colbytimm
Fixes invalid telemetry tag for ToolArea (microsoft#346)
c6292e8 
* use full toolname when finding area.

* Add overload to use an mcp client to call.

* Add test for invocation of tool

* Keep track of command names by IAreaSetup to find Service Area rather than using tokenized name.

* Fix an add tests for CommandFactory.

* Add test to verify subscription guid and status.

* Update CHANGELOG.

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix Typo.

* Move null-overriding.

* Fix Name normalization with sys commandline

* Fix test assertions from merged upstream

* Fix whitespace

* Revert change to command test

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@joshfree joshfree joshfree approved these changes

@jongio jongio jongio approved these changes

@sandeep-sen sandeep-sen sandeep-sen approved these changes

@hallipr hallipr Awaiting requested review from hallipr hallipr is a code owner automatically assigned from microsoft/azure-mcp

@KarishmaGhiya KarishmaGhiya Awaiting requested review from KarishmaGhiya KarishmaGhiya is a code owner automatically assigned from microsoft/azure-mcp

@tmeschter tmeschter Awaiting requested review from tmeschter tmeschter is a code owner automatically assigned from microsoft/azure-mcp

@xiangyan99 xiangyan99 Awaiting requested review from xiangyan99 xiangyan99 is a code owner automatically assigned from microsoft/azure-mcp

@JonathanCrd JonathanCrd Awaiting requested review from JonathanCrd JonathanCrd is a code owner automatically assigned from microsoft/azure-mcp

@fanyang-mono fanyang-mono Awaiting requested review from fanyang-mono fanyang-mono is a code owner automatically assigned from microsoft/azure-mcp

Assignees

@conniey conniey

Labels

tools-Telemetry

Projects

Azure MCP Server
Status: Done

Milestone

2025-09

Development

Successfully merging this pull request may close these issues.

5 participants

@conniey @joshfree @jongio @sandeep-sen