TestCaseFilter with custom properties #3015
Conversation
engyebrahim
changed the title
| [ArgumentsProvider(nameof(TargetFrameworks.All), typeof(TargetFrameworks))] | ||
| public async Task UsingTestPropertyForOwnerAndPriorityAndTestCategory_TestsFailed(string currentTfm) | ||
| { | ||
| var testHost = TestHost.LocateFrom(_testAssetFixture.TargetAssetPath, AssetName, currentTfm); | ||
|
|
||
| TestHostResult testHostResult = await testHost.ExecuteAsync("--filter tree!~one"); | ||
|
|
||
| testHostResult.AssertOutputContains(""" | ||
| failed PriorityTest 0ms | ||
| UTA023: TestClass: Cannot define predefined property Priority on method PriorityTest. | ||
| failed OwnerTest 0ms | ||
| UTA023: TestClass: Cannot define predefined property Owner on method OwnerTest. | ||
| failed TestCategoryTest 0ms | ||
| UTA023: TestClass: Cannot define predefined property TestCategory on method TestCategoryTest. | ||
| """); | ||
| } | ||
|
|
||
| [ArgumentsProvider(nameof(TargetFrameworks.All), typeof(TargetFrameworks))] | ||
| public async Task RunWithFilter_UsingTestPropertyForOwner_FilteredButTestsFailed(string currentTfm) | ||
| { | ||
| var testHost = TestHost.LocateFrom(_testAssetFixture.TargetAssetPath, AssetName, currentTfm); | ||
|
|
||
| TestHostResult testHostResult = await testHost.ExecuteAsync("--filter owner=testOwner"); | ||
|
|
||
| testHostResult.AssertOutputContains(""" | ||
| failed OwnerTest 0ms | ||
| UTA023: TestClass: Cannot define predefined property Owner on method OwnerTest. | ||
| """); | ||
| } | ||
|
|
||
| [ArgumentsProvider(nameof(TargetFrameworks.All), typeof(TargetFrameworks))] | ||
| public async Task RunWithFilter_UsingTestPropertyForPriorityAndTestCategory_NotFiltered(string currentTfm) | ||
| { | ||
| var testHost = TestHost.LocateFrom(_testAssetFixture.TargetAssetPath, AssetName, currentTfm); | ||
|
|
||
| TestHostResult testHostResult = await testHost.ExecuteAsync("--filter TestCategory=category|Priority=1"); | ||
|
|
||
| testHostResult.AssertOutputContains("Zero tests ran"); | ||
| } |
There was a problem hiding this comment.
What are these tests proving? The errors seem unrelated to adding a new functionality that can filter on additional test categories.
There was a problem hiding this comment.
yup it's not related to the fix, I just want to document that behaviour as it's related to TestProperty if would change it in future
There was a problem hiding this comment.
LGTM
@nohwnd I'll let you do the final approval here as you have been following more closely.
There was a problem hiding this comment.
I think this is useful functionality and it aligns well with traits on tests being used as properties of the test.
There is some slight perf impact when the property filter is enabled, but it is not the mainline scenario (discovery under VS), and we should be able to implement this in a more optimized way with ease if we drop object model from mstest.
|
Nice work @engyebrahim |
fix: #731
fix: #201