Speed up CLI argument escaping tests - #1773
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 323a2062-9493-4be4-a710-f6c2b9e15a49
There was a problem hiding this comment.
🟢 Ready to approve
The changes are isolated to test infrastructure and make the escaping tests faster while maintaining child-process coverage and adding explicit exit-code validation.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR accelerates the StdioClientTransport CLI argument escaping test suite by avoiding full MCP handshake/tool invocations and instead using a lightweight TestServer execution mode that echoes the parsed CLI argument to stderr and exits immediately, preserving real child-process invocation coverage across Windows cmd.exe, Unix dotnet, and Mono launch paths.
Changes:
- Added a
--echo-cli-arg-and-exitmode to the TestServer that writesCLI_ARG:<json>to stderr and exits. - Updated
EscapesCliArgumentsCorrectlyto capture the echoed value from stderr, JSON-parse it, and assert a clean process exit (exit code 0) via completion details.
File summaries
| File | Description |
|---|---|
| tests/ModelContextProtocol.TestServer/Program.cs | Adds an early-exit mode that echoes the parsed --cli-arg= value to stderr for fast, deterministic test verification. |
| tests/ModelContextProtocol.Tests/Transport/StdioClientTransportTests.cs | Reworks the escaping theory test to use the new echo mode + stderr capture, and asserts clean transport/process shutdown details. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Low
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
tarekgh
left a comment
There was a problem hiding this comment.
Further optimization opportunities
- Biggest remaining cost = 38 separate child-process launches (~250ms each ≈ 9-10s per framework). To go materially faster, collapse the [Theory] into a single [Fact] that launches the child once, passing all 38 values ( --cli-arg= repeated), with the echo mode emitting one CLI_ARG:: line per value. That turns 38 launches into 1 per framework (sub-second). Tradeoff: you lose per-case failure granularity, so mitigate by collecting all mismatches and reporting them together ( Assert.All over the parsed results) rather than failing on the first. The escaping logic is still fully exercised because each argument is independently escaped by the transport's argument-list builder.
- Trim the modern-TFM matrix for this theory. The escaping code differs between net472 (legacy argument-string building) and modern .NET ( ArgumentList ), so net472 + one modern TFM are the meaningful axes. net8.0/net9.0/net10.0 almost certainly run identical escaping code, so running the full 38-case theory on all three is largely redundant. Restricting the heavy theory to net-latest + net472 (leaving a smoke case on the others) roughly halves the remaining cost and is orthogonal to no. 1.
I wanted to keep the cases separate, but I don't feel strongly about it. We can always do this in a future PR.
This warrants a more holistic discussion about what tests to condition and on which TFMs. This will give the biggest perf improvement but it's at the cost of losing coverage. I think we should get the low hanging perf wins first and then consider this. |
Replace repeated MCP handshake/tool calls with a lightweight TestServer mode that echoes the parsed CLI argument through stderr and exits cleanly. This preserves real child-process coverage across Windows
cmd.exe, Unixdotnet, and Mono launch paths.Targeted runtime across all frameworks dropped from 3m 53s to ~8s (~30× faster), saving approximately 3½ minutes per affected CI leg and potentially from overall PR validation time.
Validation:
dotnet buildand all 38 targeted cases pass on net10.0, net9.0, net8.0, and net472.