-
Notifications
You must be signed in to change notification settings - Fork 6.1k
Enhance documentation for TUnit integration and usage #50094
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
0b498f8
890c2d7
8b3ae55
55dc4ba
8b21a13
a877e80
95c9bb7
29a5730
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -41,7 +41,7 @@ The main driving factors for the evolution of the new testing platform are detai | |
| * MSTest. In MSTest, the support of `Microsoft.Testing.Platform` is done via [MSTest runner](unit-testing-mstest-runner-intro.md). | ||
| * NUnit. In NUnit, the support of `Microsoft.Testing.Platform` is done via [NUnit runner](unit-testing-nunit-runner-intro.md). | ||
| * xUnit.net: In xUnit.net, the support of `Microsoft.Testing.Platform` is done via [xUnit.net runner](https://xunit.net/docs/getting-started/v3/microsoft-testing-platform). | ||
| * TUnit: entirely constructed on top of the `Microsoft.Testing.Platform`, for more information, see [TUnit documentation](https://tunit.dev/). | ||
| * TUnit: Built entirely on Microsoft.Testing.Platform and doesn't support VSTest. Unlike MSTest, NUnit, and xUnit which support both VSTest and MTP, TUnit only supports MTP. For more information, see [TUnit documentation](https://tunit.dev/) and [Getting started with TUnit](unit-testing-csharp-with-tunit.md). | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if we should skip the part
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree. I would focus more on info about TUnit rather than making it more of a comparison. |
||
|
|
||
| ## Run and debug tests | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -169,4 +169,46 @@ public void InvalidPrice_ShouldThrowExceptionWithCorrectMessage() | |
|
|
||
| :::image type="content" source="media/stryker-final-report.png" lightbox="media/stryker-final-report.png" alt-text="Stryker final report"::: | ||
|
|
||
| ## Using mutation testing with TUnit | ||
|
|
||
| Mutation testing with Stryker.NET works with TUnit. The examples above can be written using TUnit's async testing syntax: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How does it work? I thought MTP isn't yet supported by Stryker.NET stryker-mutator/stryker-net#3094 |
||
|
|
||
| ```csharp | ||
| [Test] | ||
| public async Task ApplyDiscountCorrectly() | ||
| { | ||
| decimal price = 100; | ||
| decimal discountPercent = 10; | ||
|
|
||
| var calculator = new PriceCalculator(); | ||
|
|
||
| var result = calculator.CalculatePrice(price, discountPercent); | ||
|
|
||
| await Assert.That(result).IsEqualTo(90.00m); | ||
| } | ||
|
|
||
| [Test] | ||
| public async Task InvalidDiscountPercent_ShouldThrowException() | ||
| { | ||
| var calculator = new PriceCalculator(); | ||
|
|
||
| await Assert.That(() => calculator.CalculatePrice(100, -1)) | ||
| .Throws<ArgumentException>(); | ||
| await Assert.That(() => calculator.CalculatePrice(100, 101)) | ||
| .Throws<ArgumentException>(); | ||
| } | ||
|
|
||
| [Test] | ||
| public async Task InvalidPrice_ShouldThrowExceptionWithCorrectMessage() | ||
| { | ||
| var calculator = new PriceCalculator(); | ||
|
|
||
| await Assert.That(() => calculator.CalculatePrice(0, 10)) | ||
| .Throws<ArgumentException>() | ||
| .WithMessage("Price must be greater than zero."); | ||
| } | ||
| ``` | ||
|
|
||
| For more information about TUnit, see [Unit testing C# with TUnit](unit-testing-csharp-with-tunit.md). | ||
|
|
||
| Mutation testing helps to find opportunities to improve tests that make them more reliable. It forces you to check not only the 'happy path', but also complex boundary cases, reducing the likelihood of bugs in production. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -92,6 +92,37 @@ To order tests explicitly, NUnit provides an [`OrderAttribute`](https://docs.nun | |
|
|
||
| :::code language="csharp" source="snippets/order-unit-tests/csharp/NUnit.TestProject/ByOrder.cs"::: | ||
|
|
||
| :::zone-end | ||
| :::zone pivot="tunit" | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You need to update |
||
|
|
||
| ## Order by dependency | ||
|
|
||
| TUnit provides a `[DependsOn]` attribute to control test execution order through explicit dependencies. When a test depends on another, TUnit ensures the prerequisite test completes before executing the dependent test. This approach allows you to maintain test parallelism for independent tests while enforcing order where necessary. | ||
|
|
||
| :::code language="csharp" source="snippets/order-unit-tests/csharp/TUnit.TestProject/ByDependency.cs"::: | ||
|
|
||
| ### Behavior | ||
|
|
||
| * **Failure handling** - By default, if a dependency fails, the dependent test is skipped. You can override this behavior by setting `ProceedOnFailure = true` on the `DependsOnAttribute`. | ||
|
||
| * **Accessing dependent test context** - You can retrieve data from a prerequisite test's context using the `GetTests` method on the `TestContext` object. | ||
|
||
| * **Multiple dependencies** - You can apply multiple `[DependsOn]` attributes to a single test to create complex dependency chains. | ||
|
||
|
|
||
| ### Handling method overloads | ||
|
|
||
| When depending on overloaded test methods, specify the parameter types explicitly: | ||
|
|
||
| ```csharp | ||
| [Test] | ||
| [DependsOn(nameof(SetupTest), [typeof(string), typeof(int)])] | ||
| public async Task DependentTest() | ||
| { | ||
| // This test runs after SetupTest(string, int) completes | ||
| } | ||
| ``` | ||
|
|
||
| > [!NOTE] | ||
| > While `[DependsOn]` provides ordering capabilities, tests should ideally be self-contained, isolated, and side-effect free. Reserve dependency ordering for scenarios where independent tests are impractical, such as deployment pipelines or expensive multi-step workflows. | ||
|
|
||
| :::zone-end | ||
|
|
||
| ## Next Steps | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,7 +31,8 @@ dotnet test --filter <Expression> | |
| | -------------- | -------------------- | | ||
| | MSTest | `FullyQualifiedName`<br>`Name`<br>`ClassName`<br>`Priority`<br>`TestCategory` | | ||
| | xUnit | `FullyQualifiedName`<br>`DisplayName`<br>`Traits` | | ||
| | Nunit | `FullyQualifiedName`<br>`Name`<br>`Priority`<br>`TestCategory` | | ||
| | NUnit | `FullyQualifiedName`<br>`Name`<br>`Priority`<br>`TestCategory` | | ||
| | TUnit | `FullyQualifiedName`<br>`Name`<br>`Category`<br>`Property` | | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's take this opportunity to reorder by name the test framework please.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does TUnit implement the VSTest-based filter syntax? 👀 |
||
|
|
||
| * **Operators** | ||
|
|
||
|
|
@@ -230,6 +231,63 @@ dotnet test --filter "(FullyQualifiedName~UnitTest1&TestCategory=CategoryA)|Prio | |
|
|
||
| For more information, see [TestCase filter](https://github.com/Microsoft/vstest-docs/blob/main/docs/filter.md). | ||
|
|
||
| :::zone-end | ||
| :::zone pivot="tunit" | ||
|
|
||
| ## TUnit examples | ||
|
|
||
| ```csharp | ||
| using TUnit.Core; | ||
|
|
||
| namespace TUnitNamespace | ||
| { | ||
| public class UnitTest1 | ||
| { | ||
| [Test, Property("Priority", "1"), Category("CategoryA")] | ||
| public async Task TestMethod1() | ||
| { | ||
| } | ||
|
|
||
| [Test, Property("Priority", "2")] | ||
| public async Task TestMethod2() | ||
| { | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| TUnit uses the `--treenode-filter` flag with a path-based syntax: | ||
|
|
||
| | Expression | Result | | ||
| |--|--| | ||
| | `dotnet test --treenode-filter "/*/*/*/*Method*"` | Runs tests whose method name contains `Method`. | | ||
| | `dotnet test --treenode-filter "/*/*/*/TestMethod1"` | Runs tests whose name is `TestMethod1`. | | ||
| | `dotnet test --treenode-filter "/*/TUnitNamespace/UnitTest1/*"` | Runs all tests in class `TUnitNamespace.UnitTest1`. | | ||
| | `dotnet test --treenode-filter "/**[Category=CategoryA]"` | Runs tests that are annotated with `[Category("CategoryA")]`. | | ||
| | `dotnet test --treenode-filter "/**[Priority=2]"` | Runs tests that have `[Property("Priority", "2")]`. | | ||
|
|
||
| In the code example, the `[Property]` and `[Category]` attributes can be used for filtering. | ||
|
|
||
| Examples using the conditional operators `|` and `&`: | ||
|
|
||
| To run tests that have `UnitTest1` in their class name **or** have a `Category` of `"CategoryA"`. | ||
|
|
||
| ```dotnetcli | ||
| dotnet test --treenode-filter "(/*/*/UnitTest1/*)|/**[Category=CategoryA]" | ||
| ``` | ||
|
|
||
| To run tests that are in class `UnitTest1` **and** have a `Category` of `"CategoryA"`. | ||
|
|
||
| ```dotnetcli | ||
| dotnet test --treenode-filter "(/*/*/UnitTest1/*)&/**[Category=CategoryA]" | ||
| ``` | ||
|
|
||
| To run tests that have either class `UnitTest1` **and** `Category` of `"CategoryA"` **or** have a `Property` with `"Priority"` of `"2"`. | ||
|
|
||
| ```dotnetcli | ||
| dotnet test --treenode-filter "((/*/*/UnitTest1/*)&/**[Category=CategoryA])|/**[Priority=2]" | ||
| ``` | ||
|
|
||
| :::zone-end | ||
|
|
||
| ## See also | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -295,6 +295,93 @@ After running this command, an HTML file represents the generated report. | |
|
|
||
| :::image type="content" source="media/test-report.png" lightbox="media/test-report.png" alt-text="Unit test-generated report"::: | ||
|
|
||
| ## Using code coverage with TUnit | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This whole page doesn't seem to have any framework-specific instructions. I don't feel this is the right way to document this. Instead, I prefer if this article creates clarification on instructions for VSTest and Microsoft.Testing.Platform, unrelated to any specific test framework. |
||
|
|
||
| TUnit is built on Microsoft.Testing.Platform and uses Microsoft.Testing.Extensions.CodeCoverage for code coverage. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we could link MTP and MTP CC pages here. |
||
|
|
||
| ### Creating a TUnit test project with code coverage | ||
|
|
||
| To create a TUnit test project with code coverage support: | ||
|
|
||
| ```dotnetcli | ||
| dotnet new install TUnit.Templates | ||
| dotnet new tunit -n TUnit.CodeCoverage.Test | ||
|
Comment on lines
+307
to
+308
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the template add/update global.json to set test runner as MTP? |
||
| ``` | ||
|
|
||
| Add the project reference to your class library: | ||
|
|
||
| ```dotnetcli | ||
| dotnet add TUnit.CodeCoverage.Test\TUnit.CodeCoverage.Test.csproj reference Numbers\Numbers.csproj | ||
| ``` | ||
|
|
||
| Add the Microsoft.Testing.Extensions.CodeCoverage package: | ||
|
|
||
| ```dotnetcli | ||
| dotnet add TUnit.CodeCoverage.Test package Microsoft.Testing.Extensions.CodeCoverage | ||
| ``` | ||
|
|
||
| ### TUnit test example | ||
|
|
||
| ```csharp | ||
| using TUnit.Assertions; | ||
| using TUnit.Assertions.Extensions; | ||
| using TUnit.Core; | ||
| using System.Numbers; | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Move before the other usings. |
||
|
|
||
| public class PrimeServiceTests | ||
| { | ||
| [Test] | ||
| public async Task IsPrime_InputIs1_ReturnFalse() | ||
| { | ||
| var primeService = new PrimeService(); | ||
|
|
||
| bool result = primeService.IsPrime(1); | ||
|
|
||
| await Assert.That(result).IsFalse(); | ||
| } | ||
|
|
||
| [Test] | ||
| [Arguments(2)] | ||
| [Arguments(3)] | ||
| [Arguments(5)] | ||
| [Arguments(7)] | ||
| public async Task IsPrime_PrimesLessThan10_ReturnTrue(int value) | ||
| { | ||
| var primeService = new PrimeService(); | ||
|
|
||
| bool result = primeService.IsPrime(value); | ||
|
|
||
| await Assert.That(result).IsTrue(); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Running code coverage with TUnit | ||
|
|
||
| Since TUnit requires Microsoft.Testing.Platform mode, ensure your `global.json` includes: | ||
|
|
||
| ```json | ||
| { | ||
| "test": { | ||
| "runner": "Microsoft.Testing.Platform" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Run tests with code coverage: | ||
|
|
||
| ```dotnetcli | ||
| dotnet test --coverage | ||
| ``` | ||
|
|
||
| This generates a `.coverage` file in the `TestResults` directory. To generate reports in other formats: | ||
|
|
||
| ```dotnetcli | ||
| dotnet test --coverage --coverage-output-format cobertura | ||
| ``` | ||
|
|
||
| For more information about TUnit, see [Unit testing C# with TUnit](unit-testing-csharp-with-tunit.md). | ||
|
|
||
| ## See also | ||
|
|
||
| - [Visual Studio unit test code coverage](/visualstudio/test/using-code-coverage-to-determine-how-much-code-is-being-tested) | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to the Markdown writing style guidelines, use the Oxford comma in all lists. The phrase "MSTest, NUnit, and xUnit" needs a comma before "and" (Oxford comma). Change "MSTest, NUnit and xUnit" to "MSTest, NUnit, and xUnit".