Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate to https://github.com/VerifyTests/Verify #10993

Merged
merged 2 commits into from Jun 1, 2023
Merged

Migrate to https://github.com/VerifyTests/Verify #10993

merged 2 commits into from Jun 1, 2023

Conversation

RussKie
Copy link
Member

@RussKie RussKie commented May 29, 2023

No description provided.

@ghost ghost assigned RussKie May 29, 2023
RussKie
RussKie commented May 29, 2023
View reviewed changes
UnitTests/GitCommands.Tests/Git/GetAllChangedFilesOutputParserTest.cs
}
// git status --porcelain=2 --untracked-files=no -z
IReadOnlyList<GitItemStatus> statuses = getAllChangedFilesOutputParser.Parse(statusString);
await Verifier.VerifyJson(JsonConvert.SerializeObject(statuses))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SimonCropp could you give me a hint whether I missed something here? ApprovalTests produced "proper" json, and Verify appears to produce json-like output. Is there a way to retain the proper json format? And is there a reason why VerifyJson uses .txt instead of .json for snapshots?
Thanks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u try UseStrictJson https://github.com/VerifyTests/Verify/blob/main/docs/serializer-settings.md#usestrictjson

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tip, I'll give it a try.

@mstv
Copy link
Member

mstv commented May 30, 2023

With this PR, I cannot reproduce the project loading errors in MSVS any longer. 👍

Though the test discovery fails with the same errors yet: 
Multiple source code repositories are available, using D:\Build\gitextensions3_dev as the repository root.
Test data store opened in 0,001 sec.
========== Starting test discovery ==========
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.<>c__DisplayClass27_0.<DiscoverTestsOnConcurrentManager>b__0()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.<>c__DisplayClass27_0.<DiscoverTestsOnConcurrentManager>b__0()
   at System.Threading.Tasks.Task.Execute()<---

NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
========== Test discovery aborted: 448 Tests found in 1,1 sec ==========
Test project CommonTestUtils does not reference any .NET NuGet adapter. Test discovery or execution might not work for this project.
It's recommended to reference NuGet test adapters in each test project in the solution.
Multiple source code repositories are available, using D:\Build\gitextensions3_dev as the repository root.
Test data store opened in 0,001 sec.
========== Starting test discovery ==========
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.<>c__DisplayClass27_0.<DiscoverTestsOnConcurrentManager>b__0()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.<>c__DisplayClass27_0.<DiscoverTestsOnConcurrentManager>b__0()
   at System.Threading.Tasks.Task.Execute()<---

NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery starting
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
NUnit Adapter 3.17.0.0: Test discovery complete
========== Test discovery aborted: 448 Tests found in 1,2 sec ==========

Microsoft Visual Studio Community 2022
Version 17.6.2
VisualStudio.17.Release/17.6.2+33723.286
Microsoft .NET Framework
Version 4.8.04084

Installed Version: Community

Visual C++ 2022   00482-90000-00000-AA418
Microsoft Visual C++ 2022

ASP.NET and Web Tools   17.6.326.62524
ASP.NET and Web Tools

Azure App Service Tools v3.0.0   17.6.326.62524
Azure App Service Tools v3.0.0

C# Tools   4.6.0-3.23259.8+c3cc1d0ceeab1a65da0217e403851a1e8a30086a
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

NuGet Package Manager   6.6.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

Test Adapter for Boost.Test   1.0
Enables Visual Studio's testing tools with unit tests written for Boost.Test.  The use terms and Third Party Notices are available in the extension installation directory.

Test Adapter for Google Test   1.0
Enables Visual Studio's testing tools with unit tests written for Google Test.  The use terms and Third Party Notices are available in the extension installation directory.

TypeScript Tools   17.0.20329.2001
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools   4.6.0-3.23259.8+c3cc1d0ceeab1a65da0217e403851a1e8a30086a
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools   17.6.0-beta.23174.5+0207bea1afae48d9351ac26fb51afc8260de0a97
Microsoft Visual F# Tools

Visual Studio IntelliCode   2.2
AI-assisted development for Visual Studio.

WiX Toolset Visual Studio Extension   1.0.0.22
WiX Toolset Visual Studio Extension version 1.0.0.22
Copyright (c) .NET Foundation and contributors. All rights reserved.

image

Why does MSVS not find the NUnit3TestAdapter and the NUnit.ConsoleRunner although it is referenced in the solution?

Adding explicit references results in warnings: NETSDK1023 A PackageReference for 'NUnit3TestAdapter' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs

The dotnet CLI is able to run the tests. But MSVS still cannot discover them.
$env:COREHOST_TRACE=1; $env:COREHOST_TRACE_VERBOSITY=4; dotnet test

@mstv
Copy link
Member

mstv commented May 30, 2023

dotnet test created a new empty file IntegrationTests\UI.IntegrationTests\Script\ScriptManagerTests.Can_save_settings.verified.xml (with BOM) and

IntegrationTests\UI.IntegrationTests\Script\ScriptManagerTests.Can_save_settings.received.xml:

<ArrayOfScriptInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <ScriptInfo>
    <Enabled>true</Enabled>
    <Name>name</Name>
    <Command>cmd</Command>
    <Arguments>args</Arguments>
    <AddToRevisionGridContextMenu>false</AddToRevisionGridContextMenu>
    <OnEvent>None</OnEvent>
    <AskConfirmation>false</AskConfirmation>
    <RunInBackground>false</RunInBackground>
    <IsPowerShell>false</IsPowerShell>
    <HotkeyCommandIdentifier>0</HotkeyCommandIdentifier>
    <Icon>bug</Icon>
  </ScriptInfo>
</ArrayOfScriptInfo>

@RussKie
Copy link
Member Author

RussKie commented May 31, 2023

dotnet test created a new empty file IntegrationTests\UI.IntegrationTests\Script\ScriptManagerTests.Can_save_settings.verified.xml (with BOM) and

That's probably an oversight on my part... This needs to get fixed.

@RussKie
Copy link
Member Author

RussKie commented May 31, 2023 •
edited

Why does MSVS not find the NUnit3TestAdapter and the NUnit.ConsoleRunner although it is referenced in the solution?

I would try to repair the installation. ...or reinstall it completely. Though it's not very common these days, VS sometimes can go crook after an update.
Also, try cleaning NuGet cache (i.e., C:\Users\<user>\.nuget\packages\).

@mstv
Copy link
Member

mstv commented May 31, 2023

I have deinstalled VS, rebooted and installed it again: no improvement.
Cleaning the NuGet cache replaced the test discovery errors with "No tests found to run."
Perhaps #10953 "Rename build server plugins" is missing something yet.

BTW: The first build in VS failed with:

Severity	Code	Description	Project	File	Line	Suppression State
Error	CS0006	Metadata file 'D:\Build\gitextensions3_dev\artifacts\Debug\obj\GitExtensions.Plugins.DeleteUnusedBranches\net6.0-windows\ref\GitExtensions.Plugins.DeleteUnusedBranches.dll' could not be found	DeleteUnusedBranches.Tests	D:\Build\gitextensions3_dev\UnitTests\Plugins\DeleteUnusedBranches.Tests\CSC	1	Active

I could resolve this by running dotnet build.

@gerhardol gerhardol mentioned this pull request May 31, 2023
Merged
@SimonCropp
Copy link
Contributor

can i help ?

@RussKie
Copy link
Member Author

RussKie commented Jun 1, 2023

can i help ?

Absolutely :)

@RussKie
Copy link
Member Author

RussKie commented Jun 1, 2023

It looks like after I applied your suggestion to fix json, I didn't fix all the verified files.

@RussKie RussKie merged commit da25113 into gitextensions:master Jun 1, 2023
4 checks passed
@RussKie RussKie deleted the use_verify branch June 1, 2023 04:23
@ghost ghost added this to the vNext milestone Jun 1, 2023
@RussKie RussKie mentioned this pull request Jun 1, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SimonCropp SimonCropp SimonCropp left review comments

Assignees

@RussKie RussKie

Labels
None yet
Projects
None yet
Milestone
4.2
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@RussKie @mstv @SimonCropp