Add Filter.Source integration tests in Microsoft.TestPlatform.Library.IntegrationTests#15652
Conversation
Agent-Logs-Url: https://github.com/microsoft/vstest/sessions/cd54345b-ac28-4df2-a11c-bf97609df345 Co-authored-by: Youssef1313 <31348972+Youssef1313@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/vstest/sessions/cd54345b-ac28-4df2-a11c-bf97609df345 Co-authored-by: Youssef1313 <31348972+Youssef1313@users.noreply.github.com>
| /// source-only-package (non-IS_VSTEST_REPO) surface area. | ||
| /// </summary> | ||
| [TestClass] | ||
| public class FilterExpressionWrapperIntegrationTests |
There was a problem hiding this comment.
@copilot All the tests look wrong. They are unit test on their current form. They should be integration tests. So there should be test assets that actually reference the package built locally.
There was a problem hiding this comment.
Replaced the unit-test-style file with proper integration tests in 3889cde.
The new FilterSourcePackageConsumerTests test asset (in test/TestAssets/) references the locally-built Microsoft.TestPlatform.Filter.Source NuGet package. Since it's a source-only package, the filter source files are embedded as contentFiles and compiled as internal sealed types into the consumer assembly — exactly as an end-user would consume it.
FilterSourceIntegrationTests.cs now extends AcceptanceTestBase and runs those 14 tests through vstest.console (both .NET Core and .NET Framework runners), verifying the package works end-to-end.
Agent-Logs-Url: https://github.com/microsoft/vstest/sessions/bd76cc82-b18d-4caa-949e-331115ede2e7 Co-authored-by: Youssef1313 <31348972+Youssef1313@users.noreply.github.com>
…terSourcePackageConsumerTests test asset Agent-Logs-Url: https://github.com/microsoft/vstest/sessions/0622e862-cf7d-4c75-a624-3ebe024133b3 Co-authored-by: Youssef1313 <31348972+Youssef1313@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/vstest/sessions/0622e862-cf7d-4c75-a624-3ebe024133b3 Co-authored-by: Youssef1313 <31348972+Youssef1313@users.noreply.github.com>
| public class FilterSourceIntegrationTests : AcceptanceTestBase | ||
| { | ||
| [TestMethod] | ||
| [NetCoreTargetFrameworkDataSource] |
There was a problem hiding this comment.
depends on where you want to test, this will run on .net testhost, under .net framework and .net runner. if there are any compiler conditions you probably want the opposite, where you run .net and .net framework testhost and just .net runner runner?
There was a problem hiding this comment.
yeah looks like this test simply runs the child project for isolation and we validate that all the tests there pass. We need to run this for both .net and .netfx, so the pattern described below.
All other tests here seems useless because they either re-run the same, or test that we can filter tests externally, but that is not the point here.
| } | ||
|
|
||
| [TestMethod] | ||
| [NetFullTargetFrameworkDataSource] |
There was a problem hiding this comment.
this is the same as above. if the goal is to run on both tfms for testhost then do
[NetFullTargetFrameworkDataSource(useDesktopRunner: false)]
[NetCoreTargetFrameworkDataSource(useDesktopRunner: false)]
Summary
Adds proper end-to-end integration tests for the
Microsoft.TestPlatform.Filter.SourceNuGet package toMicrosoft.TestPlatform.Library.IntegrationTests.The approach uses a dedicated test asset project (
FilterSourcePackageConsumerTests) that references the locally-built package as a NuGet dependency. Because the package is source-only (contentFiles), the filter source files are compiled into the consumer assembly asinternal sealedtypes — exactly as an end-user would consume the package. The integration tests then run that test asset through vstest.console to verify the package works end-to-end.The existing
Microsoft.TestPlatform.Filter.Source.UnitTestsproject continues to test the source-only-package (non-IS_VSTEST_REPO) API by directly linking the source files.New test asset:
FilterSourcePackageConsumerTestsLocated in
test/TestAssets/FilterSourcePackageConsumerTests/. ReferencesMicrosoft.TestPlatform.Filter.Sourceat$(PackageVersion)(the locally-built package). Contains 14 tests across two classes that exercise the embedded filter logic:FilterExpressionWrapperPackageTests(7 tests)Evaluatereturns true/false for matching/non-matching filterEvaluatereturns false when filter has a parse errorEvaluatesupports AND and OR operatorsValidForPropertiesreturns null for supported propertiesValidForPropertiesreturns unsupported property namesTestCaseFilterExpressionPackageTests(7 tests)MatchTestCasereturns true/false for matching/non-matching filterMatchTestCasereturns false when filter has a parse errorMatchTestCasesupports OR operatorTestCaseFilterValuereturns the original filter stringValidForPropertiesreturns null for supported propertiesValidForPropertiesreturns unsupported property namesIntegration tests in
FilterSourceIntegrationTests.csExtends
AcceptanceTestBaseand runsFilterSourcePackageConsumerTests.dllthrough vstest.console:FilterSourcePackage_AllTestsPassOnNetCore— verifies all 14 tests pass on .NET Core runnersFilterSourcePackage_AllTestsPassOnNetFramework— verifies all 14 tests pass on .NET Framework runnersFilterSourcePackage_FilterExpressionWrapperTestsCanBeFilteredByClassName— verifies vstest-level class-name filtering selects the 7 wrapper testsFilterSourcePackage_TestCaseFilterExpressionTestsCanBeFilteredByClassName— verifies vstest-level class-name filtering selects the 7 expression testsTesting
All tests are verified to build with 0 warnings and 0 errors against the locally-built
Microsoft.TestPlatform.Filter.Source.18.7.0-dev.nupkg.