From 79e8626f1ac87608799e55c13320f1be55df16dc Mon Sep 17 00:00:00 2001 From: Cosmin Vladutu Date: Tue, 16 Dec 2025 11:04:01 +0200 Subject: [PATCH 1/4] Enhance mutation testing documentation with new sections Added sections on interpreting mutation testing results, integrating Stryker.NET into CI/CD workflows, and customization options for mutation testing configurations. --- docs/core/testing/mutation-testing.md | 50 +++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/docs/core/testing/mutation-testing.md b/docs/core/testing/mutation-testing.md index 14be11aaf2fec..db3da385e2017 100644 --- a/docs/core/testing/mutation-testing.md +++ b/docs/core/testing/mutation-testing.md @@ -110,6 +110,56 @@ Stryker supports several types of mutations: For additional mutation types, see the [Stryker.NET: Mutations](https://stryker-mutator.io/docs/stryker-net/mutations) documentation. +## Interpret mutation testing results + +After running Stryker.NET, you'll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results: + +- **Killed**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors. + +- **Survived**: These changes weren't caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real. + +- **Timeout**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed. + +> [!NOTE] +> Don't chase a 100% mutation score. Instead, focus on high risk or business critical areas where undetected bugs would be most costly. + +## Adding mutation testing to your CI/CD workflow + +You can seamlessly integrate mutation testing into your continuous integration and delivery workflows. For instance, Stryker.NET can be configured to run within your Azure Pipelines or GitHub Actions setup, allowing you to enforce quality thresholds as part of your automated testing process. +``` + "thresholds": { + "high": 85, + "low": 65, + "break": 0 + } +``` + +## Customization + +Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customization of behavior using the _stryker-config.json_ file. +```json +{ + "ignore-mutations": [ + "ToString", + "Equals", + "GetHashCode" + ], + "ignore-methods": [ + "*Logs" + ] + "mutate": [ + "!**/Migrations/*", + "!**/*.Designer.cs" + ] +} +``` + +- **ignore-mutations**: These are the methods or expressions that you want to ignore because they are noisy, or you consider not needed based on your application logic needs. They will show up in your reports as `Ignored`. +- **ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs". +- **mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a _Migrations_ folder and all _.Designer.cs_ files (which are usually autogenerated) . + +For more information, see [Stryker: Configuration](https://stryker-mutator.io/docs/stryker-net/configuration/). + ## Incremental improvement If, after changing your code, the unit tests pass successfully, then they aren't sufficiently robust, and the mutant survived. From f2dd3f35c3c8a2e09b6b09c095546c3ce74014d0 Mon Sep 17 00:00:00 2001 From: Cosmin Vladutu Date: Tue, 16 Dec 2025 11:11:31 +0200 Subject: [PATCH 2/4] Update docs/core/testing/mutation-testing.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/core/testing/mutation-testing.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/core/testing/mutation-testing.md b/docs/core/testing/mutation-testing.md index db3da385e2017..61324347c9bc0 100644 --- a/docs/core/testing/mutation-testing.md +++ b/docs/core/testing/mutation-testing.md @@ -126,12 +126,17 @@ After running Stryker.NET, you'll receive a report that categorizes mutants as * ## Adding mutation testing to your CI/CD workflow You can seamlessly integrate mutation testing into your continuous integration and delivery workflows. For instance, Stryker.NET can be configured to run within your Azure Pipelines or GitHub Actions setup, allowing you to enforce quality thresholds as part of your automated testing process. -``` + +Add the following to your `stryker-config.json` file to set mutation score thresholds: + +```json +{ "thresholds": { "high": 85, "low": 65, "break": 0 } +} ``` ## Customization From 929f19b8d17929e880412ef58a0c8814c9dd7516 Mon Sep 17 00:00:00 2001 From: Cosmin Vladutu Date: Tue, 16 Dec 2025 11:12:19 +0200 Subject: [PATCH 3/4] Update docs/core/testing/mutation-testing.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/core/testing/mutation-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/core/testing/mutation-testing.md b/docs/core/testing/mutation-testing.md index 61324347c9bc0..88cfe17478f85 100644 --- a/docs/core/testing/mutation-testing.md +++ b/docs/core/testing/mutation-testing.md @@ -161,7 +161,7 @@ Besides setting thresholds for your pipeline, Stryker.NET offers the possibility - **ignore-mutations**: These are the methods or expressions that you want to ignore because they are noisy, or you consider not needed based on your application logic needs. They will show up in your reports as `Ignored`. - **ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs". -- **mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a _Migrations_ folder and all _.Designer.cs_ files (which are usually autogenerated) . +- **mutate**: Without this option, Stryker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a _Migrations_ folder and all _.Designer.cs_ files (which are usually autogenerated) . For more information, see [Stryker: Configuration](https://stryker-mutator.io/docs/stryker-net/configuration/). From e9c65880b9a5be285a78db66e13ded76e22e7d8d Mon Sep 17 00:00:00 2001 From: Cosmin Vladutu Date: Tue, 16 Dec 2025 11:12:38 +0200 Subject: [PATCH 4/4] Update docs/core/testing/mutation-testing.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/core/testing/mutation-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/core/testing/mutation-testing.md b/docs/core/testing/mutation-testing.md index 88cfe17478f85..02a8c41dd3b57 100644 --- a/docs/core/testing/mutation-testing.md +++ b/docs/core/testing/mutation-testing.md @@ -121,7 +121,7 @@ After running Stryker.NET, you'll receive a report that categorizes mutants as * - **Timeout**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed. > [!NOTE] -> Don't chase a 100% mutation score. Instead, focus on high risk or business critical areas where undetected bugs would be most costly. +> Don't chase a 100% mutation score. Instead, focus on high-risk or business-critical areas where undetected bugs would be most costly. ## Adding mutation testing to your CI/CD workflow