.NET: Upgrade to XUnit 3 and Microsoft Testing Platform by westey-m · Pull Request #4176 · microsoft/agent-framework

westey-m · 2026-02-23T12:32:44Z

Motivation and Context

Our integration tests are running too slowly, but with XUnit 3 and MTP we have more options for running tests in parallel.

Description

Upgrade to XUnit 3 and Microsoft Testing Platform

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the Contribution Guidelines
All unit tests pass, and I have added new tests where possible
Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot

Pull request overview

This PR upgrades the .NET test infrastructure from XUnit 2 to XUnit 3 and migrates to the Microsoft Testing Platform (MTP) to enable better test parallelization and improve integration test performance.

Changes:

Upgraded XUnit from v2 (2.9.3) to v3 (3.2.2) with MTP support
Switched code coverage from coverlet.collector to Microsoft.Testing.Extensions.CodeCoverage
Updated IAsyncLifetime implementations to use ValueTask instead of Task
Migrated test skip patterns from [Fact(Skip)] and [SkippableFact] to Assert.Skip/Assert.SkipWhen
Updated CI/CD workflows to use new MTP command-line syntax
Removed Xunit.Abstractions imports that are no longer needed

Reviewed changes

Copilot reviewed 113 out of 113 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
dotnet/Directory.Packages.props	Updated package versions for XUnit v3, xRetry, and code coverage
dotnet/tests/Directory.Build.props	Added MTP configuration properties and updated package references
dotnet/global.json	Added MTP test runner configuration
dotnet/tests/coverage.runsettings	New code coverage settings file for MTP
dotnet/tests/*/Fixture.cs (multiple)	Changed IAsyncLifetime methods from Task to ValueTask and added GC.SuppressFinalize
dotnet/tests/*Tests.cs (multiple)	Replaced [Fact(Skip)] and [SkippableFact] with Assert.Skip/Assert.SkipWhen patterns
dotnet/tests//.cs (multiple)	Removed unused Xunit.Abstractions imports
.github/workflows/dotnet-build-and-test.yml	Updated dotnet test commands to use MTP syntax with --project, --report-xunit-trx, --ignore-exit-code 8, and --filter-not-trait
dotnet/.github/skills/build-and-test/SKILL.md	Updated documentation with MTP usage examples and command syntax
dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/*.csproj	Removed Xunit.SkippableFact package reference
dotnet/tests/IntegrationTests/.csproj (multiple)	Added NoWarn for CS8793 (return type mismatch warning)

Copilot · 2026-02-24T11:32:17Z

.../AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistentStructuredOutputRunTests.cs

+    public override Task RunWithResponseFormatReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(SkipReason is not null, SkipReason!);


The condition SkipReason is not null will always be true because SkipReason is a const string. This means the test will always be skipped. If the intent is to conditionally skip the test, consider using a nullable string with a null default value, or remove the null check entirely if the test should always be skipped.

Copilot · 2026-02-24T11:32:18Z

.../AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistentStructuredOutputRunTests.cs

+    public override Task RunWithGenericTypeReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(SkipReason is not null, SkipReason!);


The condition SkipReason is not null will always be true because SkipReason is a const string. This means the test will always be skipped. If the intent is to conditionally skip the test, consider using a nullable string with a null default value, or remove the null check entirely if the test should always be skipped.

Copilot · 2026-02-24T11:32:18Z

.../AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistentStructuredOutputRunTests.cs

+    public override Task RunWithPrimitiveTypeReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(SkipReason is not null, SkipReason!);


The condition SkipReason is not null will always be true because SkipReason is a const string. This means the test will always be skipped. If the intent is to conditionally skip the test, consider using a nullable string with a null default value, or remove the null check entirely if the test should always be skipped.

Copilot · 2026-02-24T11:32:18Z

dotnet/tests/CopilotStudio.IntegrationTests/CopilotStudioRunTests.cs

+    public override Task RunWithChatMessageReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithChatMessageReturnsExpectedResultAsync();
+    }
+
+    public override Task RunWithChatMessagesReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithChatMessagesReturnsExpectedResultAsync();
+    }
+
+    public override Task RunWithNoMessageDoesNotFailAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithNoMessageDoesNotFailAsync();
+    }
+
+    public override Task RunWithStringReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithStringReturnsExpectedResultAsync();
+    }


The condition ManualVerification is not null will always be true because ManualVerification is a const string. This means these tests will always be skipped. According to the comment "Set to null to run the tests", it appears the intent is to conditionally skip. Consider changing ManualVerification to private const string? ManualVerification = "For manual verification"; to make it nullable, allowing developers to set it to null when needed.

Copilot · 2026-02-24T11:32:18Z

dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletionRunTests.cs

+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithChatMessageReturnsExpectedResultAsync();
+    }
+
+    public override Task RunWithNoMessageDoesNotFailAsync()
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithNoMessageDoesNotFailAsync();
+    }
+
+    public override Task RunWithChatMessagesReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithChatMessagesReturnsExpectedResultAsync();
+    }
+
+    public override Task RunWithStringReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithStringReturnsExpectedResultAsync();
+    }
+
+    public override Task SessionMaintainsHistoryAsync()
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);


The condition AnthropicChatCompletionFixture.SkipReason is not null will always be true because SkipReason is a const string. This means these tests will always be skipped. If the intent is to conditionally skip these tests, consider making SkipReason nullable or using a different pattern such as conditional compilation directives.

Suggested change

Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithChatMessageReturnsExpectedResultAsync();

}

public override Task RunWithNoMessageDoesNotFailAsync()

{

Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithNoMessageDoesNotFailAsync();

}

public override Task RunWithChatMessagesReturnsExpectedResultAsync()

{

Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithChatMessagesReturnsExpectedResultAsync();

}

public override Task RunWithStringReturnsExpectedResultAsync()

{

Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithStringReturnsExpectedResultAsync();

}

public override Task SessionMaintainsHistoryAsync()

{

Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);

Assert.Skip(AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithChatMessageReturnsExpectedResultAsync();

}

public override Task RunWithNoMessageDoesNotFailAsync()

{

Assert.Skip(AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithNoMessageDoesNotFailAsync();

}

public override Task RunWithChatMessagesReturnsExpectedResultAsync()

{

Assert.Skip(AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithChatMessagesReturnsExpectedResultAsync();

}

public override Task RunWithStringReturnsExpectedResultAsync()

{

Assert.Skip(AnthropicChatCompletionFixture.SkipReason!);

return base.RunWithStringReturnsExpectedResultAsync();

}

public override Task SessionMaintainsHistoryAsync()

{

Assert.Skip(AnthropicChatCompletionFixture.SkipReason!);

Copilot · 2026-02-24T11:32:19Z

...s/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletionChatClientAgentRunTests.cs

    public override Task RunWithFunctionsInvokesFunctionsAndReturnsExpectedResultsAsync()
-        => base.RunWithFunctionsInvokesFunctionsAndReturnsExpectedResultsAsync();
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithFunctionsInvokesFunctionsAndReturnsExpectedResultsAsync();
+    }

-    [Fact(Skip = AnthropicChatCompletionFixture.SkipReason)]
    public override Task RunWithInstructionsAndNoMessageReturnsExpectedResultAsync()
-        => base.RunWithInstructionsAndNoMessageReturnsExpectedResultAsync();
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithInstructionsAndNoMessageReturnsExpectedResultAsync();
+    }


The condition AnthropicChatCompletionFixture.SkipReason is not null will always be true because SkipReason is a const string. This means these tests will always be skipped. If the intent is to conditionally skip these tests, consider making SkipReason nullable or using a different pattern such as conditional compilation directives.

Copilot · 2026-02-24T11:32:20Z

dotnet/tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/SamplesValidation.cs

    }

-    async Task IAsyncLifetime.DisposeAsync()
+    async ValueTask IAsyncDisposable.DisposeAsync()


In XUnit v3, IAsyncLifetime.DisposeAsync should return ValueTask, not be implemented as IAsyncDisposable.DisposeAsync. The interface should be explicitly implemented as ValueTask IAsyncLifetime.DisposeAsync() to match the IAsyncLifetime interface definition.

Copilot · 2026-02-24T11:32:20Z

dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ConsoleAppSamplesValidation.cs

    }

-    async Task IAsyncLifetime.DisposeAsync()
+    async ValueTask IAsyncDisposable.DisposeAsync()


In XUnit v3, IAsyncLifetime.DisposeAsync should return ValueTask, not be implemented as IAsyncDisposable.DisposeAsync. The interface should be explicitly implemented as ValueTask IAsyncLifetime.DisposeAsync() to match the IAsyncLifetime interface definition.

Copilot · 2026-02-24T11:32:20Z

dotnet/tests/CopilotStudio.IntegrationTests/CopilotStudioRunStreamingTests.cs

+    public override Task RunWithChatMessageReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithChatMessageReturnsExpectedResultAsync();
+    }
+
+    public override Task RunWithChatMessagesReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithChatMessagesReturnsExpectedResultAsync();
+    }
+
+    public override Task RunWithNoMessageDoesNotFailAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithNoMessageDoesNotFailAsync();
+    }
+
+    public override Task RunWithStringReturnsExpectedResultAsync()
+    {
+        Assert.SkipWhen(ManualVerification is not null, ManualVerification!);
+        return base.RunWithStringReturnsExpectedResultAsync();
+    }


The condition ManualVerification is not null will always be true because ManualVerification is a const string. This means these tests will always be skipped. According to the comment "Set to null to run the tests", it appears the intent is to conditionally skip. Consider changing ManualVerification to private const string? ManualVerification = "For manual verification"; to make it nullable, allowing developers to set it to null when needed.

Copilot · 2026-02-24T11:32:21Z

...icChatCompletion.IntegrationTests/AnthropicChatCompletionChatClientAgentRunStreamingTests.cs

    public override Task RunWithFunctionsInvokesFunctionsAndReturnsExpectedResultsAsync()
-        => base.RunWithFunctionsInvokesFunctionsAndReturnsExpectedResultsAsync();
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithFunctionsInvokesFunctionsAndReturnsExpectedResultsAsync();
+    }

-    [Fact(Skip = AnthropicChatCompletionFixture.SkipReason)]
    public override Task RunWithInstructionsAndNoMessageReturnsExpectedResultAsync()
-        => base.RunWithInstructionsAndNoMessageReturnsExpectedResultAsync();
+    {
+        Assert.SkipWhen(AnthropicChatCompletionFixture.SkipReason is not null, AnthropicChatCompletionFixture.SkipReason!);
+        return base.RunWithInstructionsAndNoMessageReturnsExpectedResultAsync();
+    }


The condition AnthropicChatCompletionFixture.SkipReason is not null will always be true because SkipReason is a const string. This means these tests will always be skipped. If the intent is to conditionally skip these tests, consider making SkipReason nullable or using a different pattern such as conditional compilation directives.

westey-m added 2 commits February 23, 2026 12:19

Upgrade to xunit 3 and Microsoft Testing Platform

d4f27b1

Merge branch 'main' into xunit3-mtp-upgrade

043c787

markwallace-microsoft added documentation Improvements or additions to documentation .NET labels Feb 23, 2026

westey-m had a problem deploying to integration February 23, 2026 12:33 — with GitHub Actions Failure

github-actions bot changed the title ~~Upgrade to XUnit 3 and Microsoft Testing Platform~~ .NET: Upgrade to XUnit 3 and Microsoft Testing Platform Feb 23, 2026

westey-m added 2 commits February 23, 2026 16:43

Add build fixes for MTP

01785d3

Merge branch 'main' into xunit3-mtp-upgrade

5c2f2b0

westey-m had a problem deploying to integration February 23, 2026 17:05 — with GitHub Actions Error

westey-m had a problem deploying to integration February 23, 2026 17:05 — with GitHub Actions Failure

Run tests from dotnet dir

79b2888

westey-m had a problem deploying to integration February 23, 2026 17:16 — with GitHub Actions Failure

Fix path to coverage settings

3120860

westey-m had a problem deploying to integration February 23, 2026 17:34 — with GitHub Actions Failure

Fix net472 and integration disabled filter and add some logs

3e0ed44

westey-m temporarily deployed to integration February 23, 2026 18:05 — with GitHub Actions Inactive

westey-m had a problem deploying to integration February 23, 2026 18:05 — with GitHub Actions Failure

Fix coverage.runsettings path issue.

f933105

westey-m had a problem deploying to integration February 23, 2026 18:38 — with GitHub Actions Failure

westey-m temporarily deployed to integration February 23, 2026 18:38 — with GitHub Actions Inactive

Avoid failing for no-tests in a project

d7981c7

westey-m had a problem deploying to integration February 23, 2026 18:56 — with GitHub Actions Failure

Merge branch 'main' into xunit3-mtp-upgrade

bce6708

westey-m had a problem deploying to integration February 23, 2026 19:15 — with GitHub Actions Error

Fix post merge issues

0008643

westey-m temporarily deployed to integration February 23, 2026 19:19 — with GitHub Actions Inactive

westey-m had a problem deploying to integration February 23, 2026 19:19 — with GitHub Actions Failure

Fix test results coverage path

0883601

westey-m temporarily deployed to integration February 23, 2026 19:32 — with GitHub Actions Inactive

Merge branch 'main' into xunit3-mtp-upgrade

a04d279

westey-m temporarily deployed to integration February 24, 2026 11:01 — with GitHub Actions Inactive

westey-m changed the base branch from main to feature-xunit3-mtp-upgrade February 24, 2026 11:05

westey-m marked this pull request as ready for review February 24, 2026 11:25

Copilot AI review requested due to automatic review settings February 24, 2026 11:25

westey-m merged commit 6c3b9d4 into microsoft:feature-xunit3-mtp-upgrade Feb 24, 2026
20 checks passed

Copilot started reviewing on behalf of westey-m February 24, 2026 11:26 View session

Copilot AI reviewed Feb 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

.NET: Upgrade to XUnit 3 and Microsoft Testing Platform#4176

.NET: Upgrade to XUnit 3 and Microsoft Testing Platform#4176
westey-m merged 13 commits intomicrosoft:feature-xunit3-mtp-upgradefrom
westey-m:xunit3-mtp-upgrade

westey-m commented Feb 23, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

westey-m commented Feb 23, 2026

Motivation and Context

Description

Contribution Checklist

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants