Skip to content

Add regression test for MTP controller-only handshake exit path#54307

Open
Copilot wants to merge 6 commits into
mainfrom
copilot/fix-keynotfoundexception-dotnet-test
Open

Add regression test for MTP controller-only handshake exit path#54307
Copilot wants to merge 6 commits into
mainfrom
copilot/fix-keynotfoundexception-dotnet-test

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 14, 2026

Bug Fix

dotnet test (MTP path) could surface a misleading KeyNotFoundException when only the test host controller handshook and the process exited before a test host handshake. The exception obscured the real process failure and lacked explicit regression coverage after the original fix.

  • What was the bug?

    The exit path could reach assembly-completion logic for an execution id that never had AssemblyRunStarted, triggering dictionary lookup failure (KeyNotFoundException) instead of reporting handshake/process failure semantics.

  • How did you fix it?

    Added focused regression coverage in TestApplicationHandlerTests for the controller-handshake-only scenario:

    • Simulate a handshake with HostType = "TestHostController" (no TestHost handshake).
    • Invoke OnTestProcessExited(...).
    • Assert no exception is thrown and handshake failure is reported.
  • Testing

    Added a new unit/regression test case in test/dotnet.Tests/CommandTests/Test/TestApplicationHandlerTests.cs to lock this behavior.

handler.OnHandshakeReceived(
    new HandshakeMessage(new Dictionary<byte, string>
    {
        [HandshakeMessagePropertyNames.ExecutionId] = "execution-id",
        [HandshakeMessagePropertyNames.HostType] = "TestHostController",
        [HandshakeMessagePropertyNames.Framework] = ".NET 9.0.0",
        [HandshakeMessagePropertyNames.Architecture] = "x64",
    }),
    gotSupportedVersion: true);

Action act = () => handler.OnTestProcessExited(exitCode: 1, outputData: "out", errorData: "err");

act.Should().NotThrow();
output.HasHandshakeFailure.Should().BeTrue();

Copilot AI and others added 3 commits May 14, 2026 12:23
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix KeyNotFoundException in dotnet test for MTP Add regression test for MTP controller-only handshake exit path May 14, 2026
Copilot AI requested a review from Evangelink May 14, 2026 12:29
@Evangelink Evangelink marked this pull request as ready for review May 14, 2026 15:11
@Evangelink Evangelink requested review from a team as code owners May 14, 2026 15:11
Copilot AI review requested due to automatic review settings May 14, 2026 15:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds regression coverage for the MTP dotnet test controller-only handshake exit path so process exit handling reports handshake failure instead of surfacing an unrelated exception.

Changes:

  • Adds a focused TestApplicationHandlerTests test for TestHostController handshake without TestHost.
  • Includes an incidental SARIF file encoding/newline change that should be removed.
Show a summary per file
File Description
test/dotnet.Tests/CommandTests/Test/TestApplicationHandlerTests.cs Adds the regression test for controller-only handshake exit handling.
src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers.sarif Contains an unrelated generated asset encoding/newline change.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

This file was modified by build artifacts and is unrelated to the PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Member

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review using two complementary lenses: (1) general dotnet/sdk conventions, and (2) the testfx expert-reviewer 21-dimension methodology.

Verification

  • Built dotnet.Tests.csproj clean (0 warn / 0 err) on this branch.
  • Ran the new test in isolation: Total: 1, Failed: 0.
  • Traced production code in TestApplicationHandler.OnTestProcessExited (lines 270–284) and TerminalTestReporter.AssemblyRunCompleted (line 734, _assemblies[executionId]) — confirmed the test exactly exercises the HandshakeFailure branch that the original fix introduced, and would fail if that guard were removed.

Strengths

  • Single concern, good scope discipline (the unrelated NetAnalyzers.sarif drift was explicitly reverted in 16b97f0).
  • File location, file-scoped namespace, FluentAssertions + Moq usage all match TestProgressStateTests.cs and the rest of the folder.
  • Internals reachable via InternalsVisibleTo("dotnet.Tests", …) in src/Cli/dotnet/Properties/AssemblyInfo.cs. Global usings (Xunit, FluentAssertions) are leveraged correctly.

Expert-reviewer dimension summary (test-only PR — most dimensions are N/A)

# Dimension Verdict
13 Test Completeness & Coverage 🟡 1 MODERATE
17 Documentation Accuracy 🟢 1 NIT

✅ 19/21 dimensions clean (10 Test Isolation, 11 Assertion Quality, 12 Flakiness, 15 Code Structure, 16 Naming, 21 Scope all clean; the rest are N/A for a test-only change).

  • Dim 13 — Test Completeness: only the negative branch of OnTestProcessExited is covered. Add a companion [Fact] for HostType = "TestHost"AssemblyRunCompleted (no throw, HasHandshakeFailure == false) to lock in the positive branch.
  • Dim 17 — Documentation: add a one-line comment back-referencing the originating issue / fix PR (the test name carries intent, but an explicit // Regression for #… makes triage cheaper).

Pre-existing follow-up (out of scope here)
The comment block at src/Cli/dotnet/Commands/Test/MTP/TestApplicationHandler.cs:273-276 is inside the AssemblyRunCompleted branch but reads as if it belonged to the HandshakeFailure branch — its phrasing inverts the actual flow. Worth a cleanup commit later.

Verdict: No BLOCKING findings → COMMENT (not REQUEST_CHANGES). Safe to merge as-is; both findings above are quality nits.

public class TestApplicationHandlerTests
{
[Fact]
public void OnTestProcessExited_WithOnlyControllerHandshake_ReportsHandshakeFailure()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NIT] Documentation Accuracy (dim 17)

The test name carries the intent, but adding a one-line back-reference to the originating issue or fix PR makes future triage cheaper. Suggested:

// Regression for <issue/PR>: previously `OnTestProcessExited` could throw `KeyNotFoundException`
// when only a `TestHostController` handshake was received before the process exited.
[Fact]
public void OnTestProcessExited_WithOnlyControllerHandshake_ReportsHandshakeFailure()


act.Should().NotThrow();
output.HasHandshakeFailure.Should().BeTrue();
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MODERATE] Test Completeness & Coverage (dim 13)

The new test covers only the else branch of OnTestProcessExited (controller-only → HandshakeFailure). Consider adding a companion [Fact] for the positive branch so a future change that breaks the normal AssemblyRunStarted/AssemblyRunCompleted pairing is also caught here:

[Fact]
public void OnTestProcessExited_WithTestHostHandshake_ReportsAssemblyRunCompleted()
{
    // ... same setup ...
    handler.OnHandshakeReceived(
        new HandshakeMessage(new Dictionary<byte, string>
        {
            [HandshakeMessagePropertyNames.ExecutionId] = "execution-id",
            [HandshakeMessagePropertyNames.Architecture] = "x64",
            [HandshakeMessagePropertyNames.Framework] = ".NET 9.0.0",
            [HandshakeMessagePropertyNames.HostType] = "TestHost",
            [HandshakeMessagePropertyNames.InstanceId] = "instance-id",
        }),
        gotSupportedVersion: true);

    Action act = () => handler.OnTestProcessExited(exitCode: 0, outputData: "", errorData: "");

    act.Should().NotThrow();
    output.HasHandshakeFailure.Should().BeFalse();
}

This is a quality nit, not a blocker for this PR — feel free to follow up separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dotnet test for MTP throws hard-to-interpret KeyNotFoundException

3 participants