Skip to content

Add Consolidated mode#784

Merged
fanyang-mono merged 19 commits intomicrosoft:mainfrom
fanyang-mono:composite_tools
Oct 16, 2025
Merged

Add Consolidated mode#784
fanyang-mono merged 19 commits intomicrosoft:mainfrom
fanyang-mono:composite_tools

Conversation

@fanyang-mono
Copy link
Copy Markdown
Contributor

@fanyang-mono fanyang-mono commented Oct 13, 2025

What does this PR do?

Add consolidated mode, which group Azure MCP tools by tasks and actions tools conduct. This reduces the number of tools being presented to LLM, while preserves ToolMetadata values. The definition of the consolidated tools and individual commands being mapped to each tool is defined in core/Azure.Mcp.Core/src/Areas/Server/Resources/consolidated-tools.json

It honors both --read-only and --namespace switches.

Note that tools being added from external MCP servers are considered for grouping, as there is no way to get their toolMetadata. They will be loaded the same as namespace mode.

Validation:

Test result from ToolDescriptionEvaluator: Acceptable confidence (≥0.4): 87.0%

Telemetry

For consolidated mode, these are the values to be captured in telemetry event:

  • ServerMode: consolidated
  • ToolArea: <consolidated_tool_name>
  • ToolName: <individual_tool_name>

To-Do:

  • Add instructions for coding agent to add new tools to the consolidated tool structure (follow-up PR)

GitHub issue number?

N/A

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Updated servers/Azure.Mcp.Server/CHANGELOG.md and/or servers/Fabric.Mcp.Server/CHANGELOG.md for product changes (features, bug fixes, UI/UX, updated dependencies)
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes using script at eng/scripts/Process-PackageReadMe.ps1. See Package README
    • Updated command list in /servers/Azure.Mcp.Server/docs/azmcp-commands.md and/or /docs/fabric-commands.md
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated test prompts in /servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a consolidated mode for Azure MCP tools that groups related Azure operations into logical collections, reducing the number of tools presented to LLMs while preserving tool metadata. The mode aggregates individual commands by functionality (e.g., database operations, storage management) into consolidated tools that can execute multiple related operations.

  • Adds consolidated tool mode via --mode consolidated CLI option
  • Implements consolidated tool discovery strategy and server providers
  • Creates consolidated tools configuration file defining tool groupings and metadata
  • Updates metadata handling for tool consolidation

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.Sql/src/Commands/Server/ServerListCommand.cs Updates SQL server list command metadata (OpenWorld: true → false)
tools/Azure.Mcp.Tools.ManagedLustre/src/Commands/FileSystem/FileSystemCreateCommand.cs Updates filesystem create command metadata (Destructive: false → true)
tools/Azure.Mcp.Tools.KeyVault/src/Commands/Admin/AdminSettingsGetCommand.cs Updates KeyVault admin settings metadata (OpenWorld: true → false)
core/Azure.Mcp.Core/tests/Azure.Mcp.Core.UnitTests/Areas/Server/Commands/Discovery/ConsolidatedToolDiscoveryStrategyTests.cs Adds unit tests for the new consolidated tool discovery strategy
core/Azure.Mcp.Core/tests/Azure.Mcp.Core.LiveTests/Areas/Server/ServerCommandTests.cs Adds comprehensive integration tests for consolidated mode functionality
core/Azure.Mcp.Core/src/Commands/CommandGroup.cs Adds ToolMetadata property to CommandGroup for metadata preservation
core/Azure.Mcp.Core/src/Areas/Server/ServerJsonContext.cs Adds JSON serialization context for ConsolidatedToolDefinition
core/Azure.Mcp.Core/src/Areas/Server/Resources/consolidated-tools.json Large JSON configuration file defining consolidated tool groupings
core/Azure.Mcp.Core/src/Areas/Server/Options/ModeTypes.cs Adds ConsolidatedProxy mode type constant
core/Azure.Mcp.Core/src/Areas/Server/Models/ConsolidatedToolDefinition.cs Defines model for consolidated tool configuration
core/Azure.Mcp.Core/src/Areas/Server/Commands/ToolLoading/ServerToolLoader.cs Adds tool annotation handling for metadata preservation
core/Azure.Mcp.Core/src/Areas/Server/Commands/ServiceStartCommand.cs Updates mode validation to include consolidated mode
core/Azure.Mcp.Core/src/Areas/Server/Commands/ServiceCollectionExtensions.cs Adds dependency injection configuration for consolidated mode
core/Azure.Mcp.Core/src/Areas/Server/Commands/Discovery/McpServerMetadata.cs Adds ToolMetadata property to server metadata
core/Azure.Mcp.Core/src/Areas/Server/Commands/Discovery/ConsolidatedToolServerProvider.cs Implements server provider for consolidated tools
core/Azure.Mcp.Core/src/Areas/Server/Commands/Discovery/ConsolidatedToolDiscoveryStrategy.cs Main implementation of consolidated tool discovery logic

@fanyang-mono fanyang-mono requested a review from srnagar as a code owner October 13, 2025 17:53
@JasonYeMSFT
Copy link
Copy Markdown
Member

Please document how to update the consolicated-tools.json file for new tools. Ideally the documentation should also cover high level strategies such as when to fit the new tool into existing categories and when to recreate categories.

@fanyang-mono fanyang-mono requested a review from a team as a code owner October 13, 2025 20:54
@joshfree
Copy link
Copy Markdown
Member

Removing Do Not Merge as @fanyang-mono is vouching that this is very-low-risk and won't impact existing server modes like namespace.

/cc @g2vinay @anuchandy @alzimmermsft -- let's help get this in 1.0

@joshfree joshfree modified the milestones: 2025-11, 2025-10 Oct 14, 2025
@joshfree joshfree added server-Azure.Mcp Azure.Mcp.Server tools-Core AZMCP Core functionality that all other tools build on top of labels Oct 14, 2025
@fanyang-mono fanyang-mono requested a review from wbreza October 14, 2025 17:36
Comment thread core/Azure.Mcp.Core/src/Areas/Server/Models/ConsolidatedToolDefinition.cs Outdated
Comment thread tools/Azure.Mcp.Tools.Communication/src/Commands/Email/EmailSendCommand.cs Outdated
Copy link
Copy Markdown
Member

@anuchandy anuchandy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

Copy link
Copy Markdown
Member

@anuchandy anuchandy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty Fan!

@fanyang-mono fanyang-mono merged commit d44a96b into microsoft:main Oct 16, 2025
27 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Azure MCP Server Oct 16, 2025
fanyang-mono added a commit that referenced this pull request Oct 16, 2025
colbytimm pushed a commit to colbytimm/microsoft-mcp that referenced this pull request Dec 8, 2025
* Initial change for adding consolidated mode

* Make consolidated mode work end-to-end with all the tools

* Fix format

* Add all the new commands

* Merged a couple consolidated tools

* Add ToolDescriptionEvaluator result

* Add instructions on how to add new tool to consolidated mode

* Add one more tool to consolidated mode and fixed test failures

* Add documentation

* Honor --read-only server setting

* Honor --namespace server setting

* Update doc

* Update test

* Update test

* Address review feedbacks

* Fix ToolMetadata values
colbytimm pushed a commit to colbytimm/microsoft-mcp that referenced this pull request Dec 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

server-Azure.Mcp Azure.Mcp.Server tools-Core AZMCP Core functionality that all other tools build on top of

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

8 participants