Update descriptions to ensure deploy area tool invocation#2418
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adjusts the Azure Deploy tool’s command/option descriptions (and related E2E prompts) to improve tool invocation for deploy-area scenarios in Namespace and All modes (per #2245), especially around CI/CD pipeline guidance.
Changes:
- Relaxed several deploy option requirements and added defaults to reduce “missing required args” friction during tool selection/invocation.
- Updated deploy command/group titles and descriptions to better steer the model toward deploy tools for deployment plans, pipeline guidance, IaC rules, diagrams, and logs.
- Updated consolidated deploy tool metadata and E2E test prompts to better match the intended invocation phrasing.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/Azure.Mcp.Tools.Deploy/src/Options/DeployOptionDefinitions.cs | Makes several deploy options optional and sets defaults (pipeline + plan) to improve invocability. |
| tools/Azure.Mcp.Tools.Deploy/src/DeploySetup.cs | Updates deploy area command group description to emphasize deploy tool usage for pipelines/plans/constraints. |
| tools/Azure.Mcp.Tools.Deploy/src/Commands/Plan/GetCommand.cs | Refines deployment plan tool description for clearer “template/guidance” intent. |
| tools/Azure.Mcp.Tools.Deploy/src/Commands/Pipeline/GuidanceGetCommand.cs | Updates title/description to better trigger CI/CD pipeline guidance invocation. |
| tools/Azure.Mcp.Tools.Deploy/src/Commands/Infrastructure/RulesGetCommand.cs | Fixes “IaC” naming and improves description clarity for rules output. |
| tools/Azure.Mcp.Tools.Deploy/src/Commands/Architecture/DiagramGenerateCommand.cs | Updates architecture diagram tool description (but currently lists unsupported types—see comment). |
| tools/Azure.Mcp.Tools.Deploy/src/Commands/App/LogsGetCommand.cs | Marks logs tool as LocalRequired and updates description accordingly. |
| servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json | Updates deploy consolidated description and adds diagram tool to mapped tool list. |
| servers/Azure.Mcp.Server/docs/e2eTestPrompts.md | Updates deploy E2E prompts to better align with desired deploy tool invocation. |
…e-deploy-invoke
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
jongio
left a comment
There was a problem hiding this comment.
CI is red across 7/8 build platforms (only macos_arm64 passed) - that needs investigation before this can merge.
The option relaxation approach (Required=false with existing defaults) is the right fix for invocability. The title corrections (IaC, CI/CD) and LogsGetCommand LocalRequired=true are good. Diagram description now matches actual AzureServiceConstants enums after applying the Copilot suggestion.
One concern on the command group description in DeploySetup.cs - it lists the same capabilities three times in slightly different wording. This keyword-stuffing can hurt tool selection by diluting signal-to-noise ratio. A tighter single-pass description would serve the model better.
…e-deploy-invoke
…t/mcp into wc/update-deploy-invoke
There was a problem hiding this comment.
A few more items now that the description rewrites look good:
-
servers/Azure.Mcp.Server/docs/azmcp-commands.mdis stale. The metadata block fordeploy app logs getstill showsLocalRequiredas false but this PR flips it totrue, and severaldeploy plan getoptions (--target-app-service,--provisioning-tool,--source-type,--deploy-option) are shown as required even though they now have defaults. Run./eng/scripts/Update-AzCommandsMetadata.ps1per the PR checklist - this is the most likely cause of red CI. -
No changelog entry. The
LocalRequired=trueflip ondeploy_app_logs_getwill filter the tool out of HTTP/remote MCP mode (CommandFactoryToolLoader.cs:78,NamespaceToolLoader.cs:386). Combined with the Required-to-optional changes, that's user-observable behavior worth documenting perdocs/changelog-entries.md. -
The
LocalRequiredflip is the right call - the command readsazure.yamlfrom the workspace - but worth noting in the PR description so remote-server users aren't surprised when the tool disappears. -
Did you run
ToolDescriptionEvaluatorfor the new descriptions? The checklist asks for a confidence score of at least 0.4 and a top-3 ranking on each related test prompt.
The Copilot bot's diagram-description comment is stale - your current text matches AzureComputeServiceType exactly. Safe to resolve. The repetition I called out in DeploySetup.cs is also resolved in 43457c2.
jongio
left a comment
There was a problem hiding this comment.
The DeploySetup.cs description is much tighter now and the diagram description matches the actual AzureComputeServiceType / AzureServiceType enums. Both prior comments are addressed.
CI is still red on 4 of 8 build platforms on the latest commit (linux_arm64, linux_x64, macos_x64, windows_x64) - all mcp - pullrequest Azure DevOps build jobs. The macos_arm64, windows_arm64, and both musl docker builds pass. Worth investigating before merge - the failures look platform-specific rather than related to the description text changes, but the build logs will confirm.
…e-deploy-invoke
Co-authored-by: Copilot <copilot@github.com>
|
@jongio Thanks for the review. The pipeline failed because the |
jongio
left a comment
There was a problem hiding this comment.
Addresses prior feedback. The split of deploy_app_logs_get into its own namespace tool with localRequired: true is the right fix - it can't share the namespace with tools that don't need local access. Relaxing required options to use their existing defaults reduces invocation friction without changing behavior.
Co-authored-by: Copilot <copilot@github.com>
jongio
left a comment
There was a problem hiding this comment.
Addresses prior feedback. The em-dash fix in GuidanceGetCommand.cs is a clean unicode cleanup - no functional change. CI is green across all 8 platforms. Good to merge.

What does this PR do?
Per #2245, MCP tools from deploy area are having problems being invoked. This PR update the tool description as well as the E2E test prompts.
deploy_app_logs_gettool is updated to a tool withLocalRequired=truebecause it reads localazure.yaml.GitHub issue number?
[Link to the GitHub issue this PR addresses]Pre-merge Checklist
servers/Azure.Mcp.Server/README.mdand/orservers/Fabric.Mcp.Server/README.mddocumentationREADME.mdchanges running the script./eng/scripts/Process-PackageReadMe.ps1. See Package READMEToolDescriptionEvaluatorand obtained a score of0.4or more and a top 3 ranking for all related test promptsconsolidated-tools.jsonbreaking-changelabelservers/Azure.Mcp.Server/docs/azmcp-commands.md./eng/scripts/Update-AzCommandsMetadata.ps1to update tool metadata inazmcp-commands.md(required for CI)servers/Azure.Mcp.Server/docs/e2eTestPrompts.mdcrypto mining, spam, data exfiltration, etc.)/azp run mcp - pullrequest - liveto run Live Test Pipeline