Skip to content

Azure Functions description/prompt improvement#2517

Merged
anannya03 merged 5 commits into
microsoft:mainfrom
anannya03:enhance_functions_desc
May 1, 2026
Merged

Azure Functions description/prompt improvement#2517
anannya03 merged 5 commits into
microsoft:mainfrom
anannya03:enhance_functions_desc

Conversation

@anannya03
Copy link
Copy Markdown
Contributor

@anannya03 anannya03 commented Apr 28, 2026

What does this PR do?

image image image

GitHub issue number?

#2243

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

Comment thread servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves the Azure Functions tool and command descriptions (plus associated e2e prompts) to increase the likelihood that AI agents select and invoke the correct Azure Functions MCP tools, particularly in Namespace mode scenarios described in issue #2243.

Changes:

  • Updated Azure Functions command-group and command descriptions to be more directive/explicit for agent tool selection.
  • Tweaked Azure Functions Templates e2e prompts to better trigger the intended tools.
  • Added an Azure MCP Server changelog entry documenting the prompt/description improvements.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.Functions/src/FunctionsSetup.cs Updates the top-level Functions command group description to steer agents toward tool usage.
tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Rewrites TemplateGetCommand description to emphasize triggers/bindings/templates and scaffolding behavior.
tools/Azure.Mcp.Tools.Functions/src/Commands/Project/ProjectGetCommand.cs Adjusts ProjectGetCommand description to clarify when agents should call it and what it returns.
tools/Azure.Mcp.Tools.Functions/src/Commands/Language/LanguageListCommand.cs Adjusts LanguageListCommand description to emphasize authoritative language/runtime info and recommended call order.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Updates specific Azure Functions Templates prompts intended to validate tool invocation.
servers/Azure.Mcp.Server/changelog-entries/1777429555278.yaml Adds a changelog entry describing the improvements.

Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Outdated
Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Outdated
Comment thread servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Outdated
Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Outdated
Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Outdated
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few additional observations beyond what's already been flagged:

The new descriptions are 4-5x longer than typical commands in this repo (698 and 827 chars vs the usual 150-350). That's fine if it improves tool selection scores, but watch for diminishing returns - LLMs weigh early tokens more heavily, so front-loading the most distinctive keywords matters more than exhaustive listing.

The CI failure on linux_x64 native build looks unrelated to string-only changes - likely a pre-existing or flaky issue.

Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Outdated
Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Language/LanguageListCommand.cs Outdated
@anannya03 anannya03 force-pushed the enhance_functions_desc branch from ef92bf8 to 5d329dd Compare April 30, 2026 21:41
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After rebase, the whitespace bug is fixed. Two items remain:

  1. TemplateGetCommand verb form: "Lists...or generate" mixes third-person with imperative in the same sentence. Either "List...or generate" (both imperative) or "Lists...or generates" (both third-person). @alzimmermsft's earlier suggestion handles this - just flagging since the code was rebased.

  2. All CI checks pass now.

Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Template/TemplateGetCommand.cs Outdated
Comment thread tools/Azure.Mcp.Tools.Functions/src/Commands/Language/LanguageListCommand.cs Outdated
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit 5739cd0 addresses both of my earlier findings - the verb agreement in TemplateGetCommand and the redundant invocation directives in LanguageListCommand are fixed. CI is green across all 15 checks. No new issues in the updated descriptions.

@anannya03 anannya03 merged commit eddd37c into microsoft:main May 1, 2026
15 checks passed
@github-project-automation github-project-automation Bot moved this from Untriaged to Done in Azure MCP Server May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants