Refactored ToolDescriptionEvaluator code and added more options by vcolin7 · Pull Request #899 · microsoft/mcp

vcolin7 · 2025-10-21T06:14:13Z

What does this PR do?

This pull request updates the documentation for the Tool Description Evaluator to clarify and expand instructions for testing tool descriptions, server selection, and input file usage. The changes make it easier to understand how to run tests filtered by service area, use custom files and testing individual tool descriptions. It also adds support for selecting different MCP servers (Azure and Fabric) for tool/prompts dynamic loading and for providing a path to a specific executable.

Documentation improvements:

Added detailed instructions for testing tool descriptions against both Azure and Fabric MCP servers, including how to specify the server or use a custom server executable path. [1] [2]
Renamed the "Validate mode to the "Test Single Tool" and updated its documentation. [1] [2]
Expanded explanations for server selection, including precedence rules for --server vs --server-exe and automatic executable discovery. [1] [2]
Clarified how to filter tests by service area and combine area filtering with server selection. [1] [2]
Updated references and examples for test prompt files to cover both Azure and Fabric servers, and reorganized usage sections for better readability. [1] [2] [3] [4]

GitHub issue number?

Partially addresses: #450

Pre-merge Checklist

…for missing files and invalid arguments

Copilot

Pull Request Overview

This PR refactors the ToolDescriptionEvaluator to add support for testing multiple MCP servers (Azure and Fabric) and introduces a new single-tool testing mode. Key changes include:

Removed the "azmcp_" prefix from all tool names across test prompts and configuration files
Added server selection options (--server and --server-exe) to support testing Azure and Fabric MCP servers
Introduced --test-single-tool mode for testing individual tool descriptions without modifying the codebase
Improved error handling with clearer exception messages and better CI mode handling
Enhanced documentation with detailed usage examples for new features

Reviewed Changes

Copilot reviewed 6 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md	Removed "azmcp_" prefix from all tool names in test prompts
eng/tools/ToolDescriptionEvaluator/prompts.json	Removed "azmcp_" prefix from tool names in JSON configuration
eng/tools/ToolDescriptionEvaluator/README.md	Added documentation for new server selection and single-tool testing modes
eng/tools/ToolDescriptionEvaluator/Quickstart.md	Updated quickstart guide with examples of new features
eng/tools/ToolDescriptionEvaluator/Program.cs	Implemented server selection, single-tool testing, improved error handling and argument validation
Succcessful tools.txt	New file listing successful tool evaluation results

* Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting

* Add instructions on when to not use azd init (#942) * Add instrunctions on when to not use azd init * Update CHANGELOG * Address review feedback * Refactored ToolDescriptionEvaluator code and added more options (#899) * Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting * Merge release branch after `0.9.8` release (#948) * Updated CHANGELOGs for 0.9.8 release * Increment version after release (#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Adds ToolArea and ToolName to single mode telemetry. (#952) * Make registry error message more descriptive. * Set ToolArea and ToolName in single proxy mode. * Update CHANGELOG. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Updated CHANGELOGs for 0.9.9 release --------- Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add instructions on when to not use azd init (#942) * Add instrunctions on when to not use azd init * Update CHANGELOG * Address review feedback * Refactored ToolDescriptionEvaluator code and added more options (#899) * Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting * Merge release branch after `0.9.8` release (#948) * Updated CHANGELOGs for 0.9.8 release * Increment version after release (#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Adds ToolArea and ToolName to single mode telemetry. (#952) * Make registry error message more descriptive. * Set ToolArea and ToolName in single proxy mode. * Update CHANGELOG. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding CELA approved warnings in README (#955) * Adding CELA approved warnings * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removing localization in docs link --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Updating ToC of Azure MCP readme (#968) * Adding CELA approved warnings * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removing localization in docs link * Updating ToC on Readme --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix broken links in README (#979) * Add installation GIF for Azure MCP Server extension in README (2nd attempt) (#978) * Add installation GIF for Azure MCP Server extension in README * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix typo in remove-section markers in Azure.Mcp.Server README --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Re-enable smoke tests (#981) Enable smoke tests * Prepare `1.0.0` release * Fix README image reference and rename the gif (#983) Add the install_azure_mcp_server_extension.gif asset and update README to reference the new filename (use underscores instead of dashes). --------- Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Sandeep Sen <sandeepsen@microsoft.com> Co-authored-by: Vinay Gera <vigera@microsoft.com> Co-authored-by: Chris Harris <charris@microsoft.com>

* Updated CHANGELOGs for 0.9.8 release * Increment version after release (#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Prepare version `0.9.9` release (#956) * Add instructions on when to not use azd init (#942) * Add instrunctions on when to not use azd init * Update CHANGELOG * Address review feedback * Refactored ToolDescriptionEvaluator code and added more options (#899) * Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting * Merge release branch after `0.9.8` release (#948) * Updated CHANGELOGs for 0.9.8 release * Increment version after release (#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Adds ToolArea and ToolName to single mode telemetry. (#952) * Make registry error message more descriptive. * Set ToolArea and ToolName in single proxy mode. * Update CHANGELOG. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Updated CHANGELOGs for 0.9.9 release --------- Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Increment version after release (#957) Increment package version after release * update README (#977) * Prepare `1.0.0` release (#984) * Add instructions on when to not use azd init (#942) * Add instrunctions on when to not use azd init * Update CHANGELOG * Address review feedback * Refactored ToolDescriptionEvaluator code and added more options (#899) * Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting * Merge release branch after `0.9.8` release (#948) * Updated CHANGELOGs for 0.9.8 release * Increment version after release (#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Adds ToolArea and ToolName to single mode telemetry. (#952) * Make registry error message more descriptive. * Set ToolArea and ToolName in single proxy mode. * Update CHANGELOG. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding CELA approved warnings in README (#955) * Adding CELA approved warnings * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removing localization in docs link --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Updating ToC of Azure MCP readme (#968) * Adding CELA approved warnings * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removing localization in docs link * Updating ToC on Readme --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix broken links in README (#979) * Add installation GIF for Azure MCP Server extension in README (2nd attempt) (#978) * Add installation GIF for Azure MCP Server extension in README * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix typo in remove-section markers in Azure.Mcp.Server README --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Re-enable smoke tests (#981) Enable smoke tests * Prepare `1.0.0` release * Fix README image reference and rename the gif (#983) Add the install_azure_mcp_server_extension.gif asset and update README to reference the new filename (use underscores instead of dashes). --------- Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Sandeep Sen <sandeepsen@microsoft.com> Co-authored-by: Vinay Gera <vigera@microsoft.com> Co-authored-by: Chris Harris <charris@microsoft.com> * Fix NPM and NuGet smoke tests and updated VS Code plugin to NOT be a preview (#986) * Fix NPM smoke test * Fixed where NPM packages are looked up * Changed the VS Code extension to not be marked as a preview * Increment version after release (#988) Increment package version after release --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Vinay Gera <vigera@microsoft.com> Co-authored-by: Sandeep Sen <sandeepsen@microsoft.com> Co-authored-by: Chris Harris <charris@microsoft.com>

…osoft#899) * Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting

* Updated CHANGELOGs for 0.9.8 release * Increment version after release (microsoft#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Prepare version `0.9.9` release (microsoft#956) * Add instructions on when to not use azd init (microsoft#942) * Add instrunctions on when to not use azd init * Update CHANGELOG * Address review feedback * Refactored ToolDescriptionEvaluator code and added more options (microsoft#899) * Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting * Merge release branch after `0.9.8` release (microsoft#948) * Updated CHANGELOGs for 0.9.8 release * Increment version after release (microsoft#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Adds ToolArea and ToolName to single mode telemetry. (microsoft#952) * Make registry error message more descriptive. * Set ToolArea and ToolName in single proxy mode. * Update CHANGELOG. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Updated CHANGELOGs for 0.9.9 release --------- Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Increment version after release (microsoft#957) Increment package version after release * update README (microsoft#977) * Prepare `1.0.0` release (microsoft#984) * Add instructions on when to not use azd init (microsoft#942) * Add instrunctions on when to not use azd init * Update CHANGELOG * Address review feedback * Refactored ToolDescriptionEvaluator code and added more options (microsoft#899) * Renamed --validate to --test-single-tool * Refactored Program.cs to remove duplicated tool loading logic * Updated error message logic * Refactored error handling and updated exceptions thrown * Added new --server and --server-exe args and enhanced error handling for missing files and invalid arguments * Updated docs * Fixed syntax errors * Removed azmcp_ prefix from e2eTestPrompts.md * Fixed more issues * Ran TDE * Fixed parameter validation * Updated more usre-facing messages * Updated docs and --help command info * Small doc update * Removed file added by accident. * Renamed header in README * Applied some PR feedback * Applied more PR feedback * Updated logs * Fixed formatting * Merge release branch after `0.9.8` release (microsoft#948) * Updated CHANGELOGs for 0.9.8 release * Increment version after release (microsoft#946) * Increment package version after release * Add release branch to pull request pipeline --------- Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Adds ToolArea and ToolName to single mode telemetry. (microsoft#952) * Make registry error message more descriptive. * Set ToolArea and ToolName in single proxy mode. * Update CHANGELOG. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding CELA approved warnings in README (microsoft#955) * Adding CELA approved warnings * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removing localization in docs link --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Updating ToC of Azure MCP readme (microsoft#968) * Adding CELA approved warnings * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removing localization in docs link * Updating ToC on Readme --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix broken links in README (microsoft#979) * Add installation GIF for Azure MCP Server extension in README (2nd attempt) (microsoft#978) * Add installation GIF for Azure MCP Server extension in README * Update servers/Azure.Mcp.Server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix typo in remove-section markers in Azure.Mcp.Server README --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Re-enable smoke tests (microsoft#981) Enable smoke tests * Prepare `1.0.0` release * Fix README image reference and rename the gif (microsoft#983) Add the install_azure_mcp_server_extension.gif asset and update README to reference the new filename (use underscores instead of dashes). --------- Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Sandeep Sen <sandeepsen@microsoft.com> Co-authored-by: Vinay Gera <vigera@microsoft.com> Co-authored-by: Chris Harris <charris@microsoft.com> * Fix NPM and NuGet smoke tests and updated VS Code plugin to NOT be a preview (microsoft#986) * Fix NPM smoke test * Fixed where NPM packages are looked up * Changed the VS Code extension to not be marked as a preview * Increment version after release (microsoft#988) Increment package version after release --------- Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: Connie Yau <conniey@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Vinay Gera <vigera@microsoft.com> Co-authored-by: Sandeep Sen <sandeepsen@microsoft.com> Co-authored-by: Chris Harris <charris@microsoft.com>

vcolin7 added 13 commits October 20, 2025 11:11

Renamed --validate to --test-single-tool

d91168d

Refactored Program.cs to remove duplicated tool loading logic

ec065cc

Updated error message logic

90f3797

Refactored error handling and updated exceptions thrown

fc9e7c6

Added new --server and --server-exe args and enhanced error handling …

1905aff

…for missing files and invalid arguments

Updated docs

a522a5d

Fixed syntax errors

e86fb22

Removed azmcp_ prefix from e2eTestPrompts.md

3d62a3d

Fixed more issues

3f8a939

Ran TDE

eb27e9c

Fixed parameter validation

1058aac

Updated more usre-facing messages

1c73919

Updated docs and --help command info

a2e4cd5

Copilot AI review requested due to automatic review settings October 21, 2025 06:14

vcolin7 requested review from a team as code owners October 21, 2025 06:14

vcolin7 requested review from chidozieononiwu, conniey, jairmyree, jongio, msalaman, vukelich and wbreza October 21, 2025 06:14

github-project-automation Bot added this to Azure MCP Server Oct 21, 2025

github-project-automation Bot moved this to Untriaged in Azure MCP Server Oct 21, 2025

Copilot AI reviewed Oct 21, 2025

View reviewed changes

vcolin7 self-assigned this Oct 21, 2025

vcolin7 requested review from JonathanCrd, anannya03 and g2vinay October 21, 2025 06:17

vcolin7 requested a review from srnagar October 21, 2025 06:17

vcolin7 added documentation Improvements or additions to documentation tools-ToolSelection Issues related to the Tools Description Evaluator and incorrect tool selection by LLMs labels Oct 21, 2025

vcolin7 moved this from Untriaged to In Progress in Azure MCP Server Oct 21, 2025

vcolin7 added this to the 2025-10 milestone Oct 21, 2025

Small doc update

030f832

alzimmermsft reviewed Oct 21, 2025

View reviewed changes

vcolin7 added 3 commits October 21, 2025 11:22

Removed file added by accident.

5284ab7

Renamed header in README

2228375

Applied some PR feedback

62b930b

conniey reviewed Oct 21, 2025

View reviewed changes

Comment thread eng/tools/ToolDescriptionEvaluator/Program.cs

vcolin7 added 4 commits October 21, 2025 15:34

Applied more PR feedback

af5f240

Updated logs

6ddb047

Merge remote-tracking branch 'origin/main' into tde/test-single-tool

9f84eb2

Fixed formatting

b56aaba

alzimmermsft approved these changes Oct 23, 2025

View reviewed changes

Merge remote-tracking branch 'origin/main' into tde/test-single-tool

6dd21a2

alzimmermsft approved these changes Oct 24, 2025

View reviewed changes

vcolin7 merged commit 9cdefa4 into microsoft:main Oct 24, 2025
23 checks passed

github-project-automation Bot moved this from In Progress to Done in Azure MCP Server Oct 24, 2025

vcolin7 deleted the tde/test-single-tool branch January 9, 2026 22:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactored ToolDescriptionEvaluator code and added more options#899

Refactored ToolDescriptionEvaluator code and added more options#899
vcolin7 merged 22 commits intomicrosoft:mainfrom
vcolin7:tde/test-single-tool

vcolin7 commented Oct 21, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

vcolin7 commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

GitHub issue number?

Pre-merge Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vcolin7 commented Oct 21, 2025 •

edited

Loading