Cannot run tests anymore after upgrading to 2.2.1 #789
Comments
|
A very simple sample project that reproduces the issue: UnitTestProject1.zip. To get it to execute, just comment out DeploymentItem in UnitTest1.cs. |
|
I am also having issues successfully executing tests when upgrading from v2.1.2 to v2.2.1 (I am also using DeploymentItem mentioned above) 👍 |
|
Same for me. |
|
You need to install this package Microsoft.TestPlatform.AdapterUtilities @Haplois we need to think how to prevent this issue. Everyone who update MsTestAdapter to 2.2.1 will get this error. |
|
@Sanan07 for the following project, the installation of the package did not help: |
|
Hi @avivanoff and @mprokopy Thank you for reporting this bug. There is an issue with the <RunSettings>
<RunConfiguration>
<DisableAppDomain>True</DisableAppDomain>
</RunConfiguration>
</RunSettings> |
|
@Haplois, when do you expect to have a fix? |
|
We should have a preview in a week or two max, and if everything goes well, we can do another minor release. |
|
@Haplois, is it a recommended approach to disable app domains in test execution? |
|
@avivanoff It should be fine, especially if tests are running in isolation. |
|
Hello, We have several test projects in the solution. I can run only some of them. For the others I got this errors in the test output:
Sorry for the german error message, that should be something like: Incorrect format for TestCaseFilter Error: Missing ... Ciao, |
|
@xantiva, can you clean your solution, close Visual Studio, reopen it, rebuild your entire solution and try running your tests again? Do you still get the error? |
|
@Haplois I can also reproduce the issue and restarting VS after updating to In my case, I have two test projects: the first is targeting .NET Framework 4.8 with an old-style When I update either test project project and try to run tests I get his output: The output for my I also observed the same error as the OP a few times - it was repeated for every test method in my .NET Core 3.1 Test project - but I can't seem to reproduce it anymore - which is weird. But it was something like this:
Both of my test projects have a package-reference to I also sometimes see this output: ...which is odd, because I thought it did. If I do a clean rebuild this message doesn't reappear. Update:After fiddling around with NuGet packages, I saw that if I updated |
|
@Jehoel, thanks for the verification. You can use #789 (comment) as a temporary workaround until we develop and test the fix. |
|
@Haplois I've updated my earlier reply with more details and a more reliable repro of the other error. Unfortunately the workaround you suggest won't work for me because I depend on AppDomains in the .NET Framework tests. I'll just stick with |
|
@Haplois here are my results: The target framework of the solution is .NET Core 3.1. Starting with MSTest.TestAdapter v2.1.2
|
|
Many thanks for the quick preview release. I can run unit tests with Will wait for 2.2.2 final release. Hopefully others who have reported this can verify it too. Update: Oops just realised I cross posted, please refer to #793 for details of Preview release. |
ghost
closed this as
|
@sumitkm, thanks for the verification. |
|
This bug is fixed in version |
|
@Haplois, thanks a lot. After updating to this preview version, I can run again my unit test! |
|
@Haplois, when do you expect the official release with the fixes? |
|
@avivanoff testing on this is still continuing; we can do a servicing release on Monday if everything goes well. |
|
@Haplois, any updates on the release? |
|
I preparing the release; I'll release it today. |
|
The version |
|
@Haplois, just updated to 2.2.2, the problem is not fixed. See updated sample project UnitTestProject1.zip. Please, re-open the bug. |
|
Maybe this is an other problem? I can run my tests again after updating to v2.2.2. So my problem is solved with this release. |
|
Can't run tests after update to 2.2.2. Please check the project from my post above. #789 (comment) |
|
@mprokopy, @avivanoff I missed the issue about |
|
Let's do this again :) a fixed version is just released. Please use |
Everything is fine on my side. |
|
@Haplois - is part if this fix to report failures or at least some kind of inconclusive result for the tests we instructed the tool to run but it was not able to for whatever reason? The fact that I commanded the program to do something and it failed to do that thing, and also failed to tell me there was a problem doing that thing seems like an obvious problem that should be fixed in any computer program, but I dont see that called out here. |
|
@StingyJack, the problem was caused by the parsing of test case filters, which has been addressed by the fix. Could you clarify if you're still having this issue? |
|
I don't think the specific cause actually matters. It's a giant defect if any one part of a test framework can cause the rest to stop executing without reporting a single test failure |
|
That error prevented test discovery, So all the tests discovered on previous runs or via source-based discovery would show not run status. You can configure Test Platform to report failure when no tests are discovered, but that's not the default behavior. See the documentation—example below. <?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<RunConfiguration>
<!-- true or false -->
<!-- Value that specifies the exit code when no tests are discovered -->
<TreatNoTestsAsError>true</TreatNoTestsAsError>
</RunConfiguration>
</RunSettings> |
|
@Haplois There is a clear difference between "report an error when no tests are discovered" and "the tests that show in the test explorer list are not being executed when commanded to do so, and are not reporting any failure for the tests or any other failure to perform the command in its entirety." If the test is in a Not Run state, then it has been discovered. The test is in the Test Explorer listing, I see it there, so it must be discovered at that point and known to the test runner. I tell VS to run that test and it tries to run it, but still it says "Not Run" (with the broken version of this package). Similarly the other tests in the list show as Success or Failure but they never change either. I expect them to at least be set to Not Run if I had selected them to run as part of the invocation and they did not run for whatever reason. In no way is it useful to anyone for them to continue to show the last completed run status. What this leads to is that over the course of a few hours or days I would make changes to the target code and then Run all Tests and they would all show "Success" in test explorer and in the status bar. I believe that the refactoring changes I made were good, and then made more refactorings and other changes that needed to be made that depended on some refactoring. Except that the tests I commanded to be run werent actually being run. They were showing the status of the last time they were able to run. A gross misrepresentation of the actual state that was discovered late in the development effort that requires reworking and retesting the entire UserStory task that was in progress. The summary of this is that your tool failed and did not report it to the operator. Like a train off the rails and the conductor is telling everyone "everything is fine and good and on schedule". |
|
@StingyJack, a discovered test doesn't necessarily mean that it was discovered by the test runner. It might have been discovered by the Test Explorer using source-based discovery or can be loaded from the cache. Your concerns are valid, but the Test Explorer window is not part of this repo. Please report it inside the Visual Studio, using Send Feedback tool. |
What does the adapter do? What is it responsible for? |
Attempt to upgrade our test projects to version from 2.1.2 to 2.2.1 results in the following test execution error:
An exception occurred while invoking executor 'executor://mstestadapter/v2': Could not load file or assembly 'Microsoft.TestPlatform.AdapterUtilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.AB#1287803
The text was updated successfully, but these errors were encountered: