Azure Functions description/prompt improvement#2517
Conversation
There was a problem hiding this comment.
Pull request overview
Improves the Azure Functions tool and command descriptions (plus associated e2e prompts) to increase the likelihood that AI agents select and invoke the correct Azure Functions MCP tools, particularly in Namespace mode scenarios described in issue #2243.
Changes:
- Updated Azure Functions command-group and command descriptions to be more directive/explicit for agent tool selection.
- Tweaked Azure Functions Templates e2e prompts to better trigger the intended tools.
- Added an Azure MCP Server changelog entry documenting the prompt/description improvements.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/Azure.Mcp.Tools.Functions/src/FunctionsSetup.cs | Updates the top-level Functions command group description to steer agents toward tool usage. |
| tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs | Rewrites TemplateGetCommand description to emphasize triggers/bindings/templates and scaffolding behavior. |
| tools/Azure.Mcp.Tools.Functions/src/Commands/Project/ProjectGetCommand.cs | Adjusts ProjectGetCommand description to clarify when agents should call it and what it returns. |
| tools/Azure.Mcp.Tools.Functions/src/Commands/Language/LanguageListCommand.cs | Adjusts LanguageListCommand description to emphasize authoritative language/runtime info and recommended call order. |
| servers/Azure.Mcp.Server/docs/e2eTestPrompts.md | Updates specific Azure Functions Templates prompts intended to validate tool invocation. |
| servers/Azure.Mcp.Server/changelog-entries/1777429555278.yaml | Adds a changelog entry describing the improvements. |
jongio
left a comment
There was a problem hiding this comment.
A few additional observations beyond what's already been flagged:
The new descriptions are 4-5x longer than typical commands in this repo (698 and 827 chars vs the usual 150-350). That's fine if it improves tool selection scores, but watch for diminishing returns - LLMs weigh early tokens more heavily, so front-loading the most distinctive keywords matters more than exhaustive listing.
The CI failure on linux_x64 native build looks unrelated to string-only changes - likely a pre-existing or flaky issue.
ef92bf8 to
5d329dd
Compare
jongio
left a comment
There was a problem hiding this comment.
After rebase, the whitespace bug is fixed. Two items remain:
-
TemplateGetCommand verb form: "Lists...or generate" mixes third-person with imperative in the same sentence. Either "List...or generate" (both imperative) or "Lists...or generates" (both third-person). @alzimmermsft's earlier suggestion handles this - just flagging since the code was rebased.
-
All CI checks pass now.
jongio
left a comment
There was a problem hiding this comment.
Commit 5739cd0 addresses both of my earlier findings - the verb agreement in TemplateGetCommand and the redundant invocation directives in LanguageListCommand are fixed. CI is green across all 15 checks. No new issues in the updated descriptions.
What does this PR do?
GitHub issue number?
#2243
Pre-merge Checklist
servers/Azure.Mcp.Server/README.mdand/orservers/Fabric.Mcp.Server/README.mddocumentationREADME.mdchanges running the script./eng/scripts/Process-PackageReadMe.ps1. See Package READMEToolDescriptionEvaluatorand obtained a score of0.4or more and a top 3 ranking for all related test promptsconsolidated-tools.jsonbreaking-changelabelservers/Azure.Mcp.Server/docs/azmcp-commands.md./eng/scripts/Update-AzCommandsMetadata.ps1to update tool metadata inazmcp-commands.md(required for CI)servers/Azure.Mcp.Server/docs/e2eTestPrompts.mdcrypto mining, spam, data exfiltration, etc.)/azp run mcp - pullrequest - liveto run Live Test Pipeline