Skip to content

Add per-tool JSON schema validation tests for MCP responses #1820

Description

@Widthdom

Summary

McpServerTests and CliJsonSerializerContextTests cover JSON serialization at the type-registration level (does the JSON source generator know about the type?) but do not assert the response shape per MCP tool: which fields are present, which are required, what their value types are, what naming convention they follow. A subtle rename in the response model (e.g. totaltotal_count) currently passes CI but breaks every downstream MCP client. The just-filed #1808 (truncated/total envelope) and #1810 (tool examples) make a per-tool schema contract more pressing.

Where

  • tests/CodeIndex.Tests/McpServerTests.cs

Suggested approach

(1) Define a JSON schema (*.schema.json) per MCP tool response under tests/CodeIndex.Tests/Schemas/, covering search, references, callers, callees, definition, impact_analysis, batch_query, find_in_file, excerpt, hotspots, deps, map, outline, inspect, completions, status, unused, validate. (2) Add a test that drives each tool with a canonical input and validates the response against its schema using NJsonSchema or JsonSchema.Net. (3) Treat the schemas as the contract: changing them requires a CHANGELOG entry. (4) Have the schemas auto-publish under --mcp-schema-dump so external consumers can pin against them (cross-link with #1554 — protocol version pinning). (5) Cross-link with #1808 (envelope), #1810 (examples), #1811 (language docs) — same family of contract gaps.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions