Skip to content

Add TestCategories property to ITestDataRow for per-test-case categorization #5795

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Jun 17, 2025

This PR adds support for applying test categories to individual test data rows in parameterized tests, addressing a long-standing user request for more granular test categorization.

Changes

Core API Changes

  • Added TestCategories property to ITestDataRow interface
  • Implemented TestCategories property in TestDataRow<T> class with [DataMember] attribute
  • Property is nullable to maintain backward compatibility

Discovery Logic Updates

  • Updated TestDataSourceHelpers.TryHandleITestDataRow() to extract test categories from test data rows
  • Added backward-compatible overload to maintain existing API
  • Modified discovery logic in AssemblyEnumerator to merge test categories from test data rows with existing method/class/assembly level categories

Usage Example

[TestMethod]
[DynamicData(nameof(GetTestData), DynamicDataSourceType.Method)]
public void MyParameterizedTest(string input, string expected)
{
    Assert.AreEqual(expected, ProcessInput(input));
}

public static IEnumerable<object[]> GetTestData()
{
    // Fast unit test case
    yield return new TestDataRow<(string, string)>(("input1", "output1"))
    {
        TestCategories = new List<string> { "Unit", "Fast" },
        DisplayName = "Fast unit test"
    };
    
    // Slow integration test case
    yield return new TestDataRow<(string, string)>(("input2", "output2"))
    {
        TestCategories = new List<string> { "Integration", "Slow" },
        DisplayName = "Integration test"
    };
}

Benefits

  • Granular filtering: Can now filter tests by categories applied to individual test cases
  • Mixed scenarios: Single test method can contain both unit tests and integration tests with appropriate categories
  • Category inheritance: Test data row categories are merged with method/class/assembly level categories
  • Backward compatibility: Existing code continues to work unchanged

Test Coverage

  • Added comprehensive unit tests for TestDataRow and TestDataSourceHelpers
  • Added integration tests to verify end-to-end functionality
  • Created sample test project demonstrating real-world usage
  • Validated category filtering works correctly with test discovery

This enables scenarios like:

# Run only fast test cases
dotnet test --filter "TestCategory=Fast"

# Run only integration test cases  
dotnet test --filter "TestCategory=Integration"

Fixes #4089.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • 5rqvsblobprodcus385.vsblob.vsassets.io
    • Triggering command: /home/REDACTED/work/testfx/testfx/.dotnet/dotnet msbuild /m /nologo /clp:Summary /v:quiet /nr:true /warnaserror /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=false /home/REDACTED/work/testfx/testfx/artifacts/toolset/restore.proj /t:__WriteToolsetLocation /clp:ErrorsOnly;NoSummary /p:__ToolsetLocationOutputFile=/home/REDACTED/work/testfx/testfx/artifacts/toolset/10.0.0-beta.25316.2.txt (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Copilot AI and others added 3 commits June 17, 2025 08:12
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
…ng categories

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
…feature

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] Add TestCategories to ITestDataRow Add TestCategories property to ITestDataRow for per-test-case categorization Jun 17, 2025
@Copilot Copilot AI requested a review from Evangelink June 17, 2025 08:21
Copilot finished work on behalf of Evangelink June 17, 2025 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add TestCategories to ITestDataRow
2 participants