-
Notifications
You must be signed in to change notification settings - Fork 28
Add MCP integration tests with JSON validation for all server tools #4268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…tools Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds comprehensive integration tests for MCP server tools to validate JSON output format. The new test file ensures each MCP server tool returns valid, parseable JSON with expected field structures.
Key Changes:
- Adds
mcp_server_json_integration_test.gowith JSON validation tests for all 7 MCP server tools - Implements helper functions for JSON extraction and validation from console-formatted output
- Includes graceful error handling for tools that may fail in test environments without credentials
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| // The audit tool may return an error message for invalid run IDs | ||
| // We check if the output contains "Error:" which indicates an error message | ||
| if len(textContent.Text) >= 6 && textContent.Text[0:6] == "Error:" { |
Copilot
AI
Nov 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use strings.HasPrefix(textContent.Text, \"Error:\") instead of manual bounds checking and substring comparison. This is the idiomatic Go pattern used consistently throughout the codebase (seen in 100+ locations) and is more readable and safer.
| } | ||
|
|
||
| // The logs tool may return an error message in test environment | ||
| if len(textContent.Text) >= 6 && textContent.Text[0:6] == "Error:" { |
Copilot
AI
Nov 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use strings.HasPrefix(textContent.Text, \"Error:\") instead of manual bounds checking and substring comparison. This is the idiomatic Go pattern used consistently throughout the codebase (seen in 100+ locations) and is more readable and safer.
| // If tool is expected to return JSON, validate it | ||
| if tc.expectJSON { | ||
| // Skip JSON validation if output starts with "Error:" | ||
| if len(textContent.Text) >= 6 && textContent.Text[0:6] == "Error:" { |
Copilot
AI
Nov 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use strings.HasPrefix(textContent.Text, \"Error:\") instead of manual bounds checking and substring comparison. This is the idiomatic Go pattern used consistently throughout the codebase (seen in 100+ locations) and is more readable and safer.
| import ( | ||
| "context" | ||
| "encoding/json" | ||
| "os" | ||
| "os/exec" | ||
| "path/filepath" | ||
| "testing" | ||
| "time" | ||
|
|
||
| "github.com/githubnext/gh-aw/pkg/testutil" | ||
| "github.com/modelcontextprotocol/go-sdk/mcp" | ||
| ) |
Copilot
AI
Nov 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing strings package import needed for the suggested strings.HasPrefix() usage to replace manual substring checks.
Missing integration tests for MCP server tools left JSON output format unvalidated. This adds comprehensive test coverage ensuring each tool returns valid, parseable JSON.
Changes
New test file:
pkg/cli/mcp_server_json_integration_test.goHelper functions:
isValidJSON()- JSON structure validationextractJSONFromOutput()- Strips console formatting (Unicode chars) from outputsetupMCPServerTest()- Common test harness with--cmdflag to avoid gh extension requirementIntegration tests:
status,compile,audit,logstools validating JSON structure and expected fieldsjqfilter functionality testKey handling:
✓ .github/workflows/test.md\n[{...}]) extracted correctlyExample
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.