Enhance descriptions for Kusto tools#666
Merged
vcolin7 merged 8 commits intomicrosoft:mainfrom Oct 2, 2025
Merged
Conversation
…y the previous change
vcolin7
added
tools-Kusto
tools-ToolSelection
vcolin7
requested review from
JonathanCrd,
KarishmaGhiya,
hallipr,
joshfree,
msalaman,
vukelich and
xiangyan99
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR enhances the descriptions for Kusto tools to improve their discoverability and reduce ambiguity when selected by LLMs, aiming to achieve a cosine similarity threshold of 0.4 or higher.
- Updated descriptions for all 7 Kusto command tools to include synonyms like "Azure Data Explorer" and "KQL"
- Modified parameter formatting from backticks to double dashes for consistency
- Updated changelog and test configuration files to reflect the changes
Reviewed Changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tools/Azure.Mcp.Tools.Kusto/src/Commands/TableSchemaCommand.cs | Enhanced description with synonyms and standardized parameter format |
| tools/Azure.Mcp.Tools.Kusto/src/Commands/TableListCommand.cs | Updated description to include Azure Data Explorer/KQL terms |
| tools/Azure.Mcp.Tools.Kusto/src/Commands/SampleCommand.cs | Enhanced description with synonyms and parameter formatting |
| tools/Azure.Mcp.Tools.Kusto/src/Commands/QueryCommand.cs | Improved description with more specific use cases and synonyms |
| tools/Azure.Mcp.Tools.Kusto/src/Commands/DatabaseListCommand.cs | Updated description with Azure Data Explorer/KQL terms |
| tools/Azure.Mcp.Tools.Kusto/src/Commands/ClusterListCommand.cs | Enhanced description with synonyms |
| tools/Azure.Mcp.Tools.Kusto/src/Commands/ClusterGetCommand.cs | Improved description with clarification and synonyms |
| servers/Azure.Mcp.Server/CHANGELOG.md | Added changelog entry for Kusto command description updates |
| eng/tools/ToolDescriptionEvaluator/tools.json | Updated tool descriptions in evaluation configuration |
| eng/tools/ToolDescriptionEvaluator/prompts.json | Minor formatting and ordering changes |
| docs/e2eTestPrompts.md | Fixed table formatting alignment |
anannya03
approved these changes
Oct 1, 2025
colbytimm
pushed a commit
to colbytimm/microsoft-mcp
that referenced
this pull request
Dec 8, 2025
* Updated low-scoring Kusto command descriptions * Updated the cluster get command description reduce ambiguity caused by the previous change * Updated CHANGELOG * Fixed e2eTestPrompts.md * Ran tests * Updated all Kusto descriptions to mention Azure Data Explorer/Kusto/KQL * Updated results * Updated CHANGELOG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR updates the descriptions of Kusto tools to get them all above the 0.4 cosine similarity threshold, make them the top choice and reduce ambiguity vs. the second choice.
GitHub issue number?
#253
Pre-merge Checklist
servers/Azure.Mcp.Server/CHANGELOG.mdand/orservers/Fabric.Mcp.Server/CHANGELOG.mdfor product changes (features, bug fixes, UI/UX, updated dependencies)servers/Azure.Mcp.Server/README.mdand/orservers/Fabric.Mcp.Server/README.mddocumentation/docs/azmcp-commands.mdand/or/docs/fabric-commands.mdToolDescriptionEvaluatorand obtained a score of0.4or more and a top 3 ranking for all related test prompts/docs/e2eTestPrompts.mdcrypto mining, spam, data exfiltration, etc.)/azp run mcp - pullrequest - liveto run Live Test Pipeline