Skip to content

MTP2 TRX extension cannot handle multiple tests with same name (regression from vstest) #7442

@MaceWindu

Description

@MaceWindu

Describe the bug

It is a bit hard to tell who to blame - new trx extension or nunit here, so I will start from this repo.
Test below generates valid TRX with vstest and fails with exception on MTP2

Version used

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
<PackageReference Include="Microsoft.Testing.Extensions.TrxReport" Version="2.1.0" />
<PackageReference Include="NUnit" Version="4.5.0" />
<PackageReference Include="NUnit3TestAdapter" Version="6.1.0" />

Steps To Reproduce

Actual test has more complex parameters, which are not same but produce same test name due to value to string conversions.

public class Duplicates
{
    [Test]
    public void Test([Values("one", "one")] string value)
    {
    }
}

Run

TrxIssue.exe --report-trx

Expected behavior

work as before

dotnet test TrxIssue.dll /logger:trx

TRX fragment that shows:

  • one test definition based on test name
  • two test executions
<Results>
    <UnitTestResult executionId="3fcd2166-3142-4e69-95b3-2a543de603a8" testId="3f08becb-9273-a477-1b52-2406ec116de6" testName="Test(&quot;one&quot;)"  ... testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="3fcd2166-3142-4e69-95b3-2a543de603a8" />
    <UnitTestResult executionId="b60e8ab0-7f92-4c35-ab00-fa44c74d4b57" testId="3f08becb-9273-a477-1b52-2406ec116de6" testName="Test(&quot;one&quot;)" ... testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="b60e8ab0-7f92-4c35-ab00-fa44c74d4b57" />
  </Results>
  <TestDefinitions>
    <UnitTest name="Test(&quot;one&quot;)" storage="..." id="3f08becb-9273-a477-1b52-2406ec116de6">
      <Execution id="3fcd2166-3142-4e69-95b3-2a543de603a8" />
      <TestMethod codeBase="..." adapterTypeName="executor://nunit3testexecutor/" className="Duplicates" name="Test(&quot;one&quot;)" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="3f08becb-9273-a477-1b52-2406ec116de6" executionId="3fcd2166-3142-4e69-95b3-2a543de603a8" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="3f08becb-9273-a477-1b52-2406ec116de6" executionId="b60e8ab0-7f92-4c35-ab00-fa44c74d4b57" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>

Actual behavior

[√2/x0/↓0] TrxIssue.dll (net10.0|x64)                                                                                                                                                 (0s)
Unhandled exception. System.ArgumentException: An item with the same key has already been added. Key: TestNodeUid { Value = Duplicates.Test("one") }
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Microsoft.Testing.Platform.Services.TestApplicationResult.ConsumeAsync(IDataProducer dataProducer, IData value, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs:line 123
   at Microsoft.Testing.Platform.Messages.AsyncConsumerDataProcessor.ConsumeAsync() in /_/src/Platform/Microsoft.Testing.Platform/Messages/AsyncConsumerDataProcessor.net.cs:line 73
   at Microsoft.Testing.Platform.Messages.AsyncConsumerDataProcessor.DrainDataAsync() in /_/src/Platform/Microsoft.Testing.Platform/Messages/AsyncConsumerDataProcessor.net.cs:line 145
   at Microsoft.Testing.Platform.Messages.AsynchronousMessageBus.DrainDataAsync() in /_/src/Platform/Microsoft.Testing.Platform/Messages/AsynchronousMessageBus.cs:line 183
   at Microsoft.Testing.Platform.Messages.MessageBusProxy.DrainDataAsync() in /_/src/Platform/Microsoft.Testing.Platform/Messages/MessageBusProxy.cs:line 37
   at Microsoft.Testing.Platform.Hosts.CommonHost.NotifyTestSessionEndAsync(ITestSessionContext testSessionContext, BaseMessageBus baseMessageBus, ServiceProvider serviceProvider, IPlatformOpenTelemetryService otelService) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 228
   at Microsoft.Testing.Platform.Hosts.CommonHost.ExecuteRequestAsync(ProxyOutputDevice outputDevice, ITestSessionContext testSessionInfo, ServiceProvider serviceProvider, BaseMessageBus baseMessageBus, ITestFramework testFramework, ClientInfo client) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 171
   at Microsoft.Testing.Platform.Hosts.ConsoleTestHost.InternalRunAsync(CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.cs:line 85
   at Microsoft.Testing.Platform.Hosts.ConsoleTestHost.InternalRunAsync(CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.cs:line 117
   at Microsoft.Testing.Platform.Hosts.CommonHost.RunTestAppAsync(IPlatformOpenTelemetryService platformOTelService, CancellationToken testApplicationCancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 130
   at Microsoft.Testing.Platform.Hosts.CommonHost.RunAsync() in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 44
   at Microsoft.Testing.Platform.Hosts.CommonHost.RunAsync() in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 83
   at Microsoft.Testing.Platform.Builder.TestApplication.RunAsync() in /_/src/Platform/Microsoft.Testing.Platform/Builder/TestApplication.cs:line 223
   at xxx.MicrosoftTestingPlatformEntryPoint.Main(String[] args)

Additional context

Metadata

Metadata

Assignees

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions