Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues with NUnit #449

Closed
boldt opened this issue May 29, 2019 · 12 comments
Closed

Issues with NUnit #449

boldt opened this issue May 29, 2019 · 12 comments
Labels
needs more info More details are needed

Comments

@boldt
Copy link

boldt commented May 29, 2019

I cannot get coverlet running. I added via NuGet:

  • NUnit: 3.12.0
  • NUnit.ConsoleRunner: 3.10.0
  • NUnit3TestAdapter: 3.13.0
  • coverlet.msbuild: 2.6.1

I ran my build and test as follows:

dotnet build
dotnet test --no-build /p:CollectCoverage=true

I'm getting the "strong name" issue, see #338 (partly German log):

[xUnit.net 00:00:00.78] Test: Catastrophic failure: System.IO.FileLoadException: Die Datei oder Assembly "xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c" oder eine Abhängigkeit davon wurde nicht gefunden. Die Signatur mit starkem Namen konnte nicht überprüft werden. Die Assembly wurde möglicherweise geändert oder verzögert aber nicht vollständig mit dem richtigen privaten Schlüssel signiert. (Ausnahme von HRESULT: 0x80131045)
Dateiname: "xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c"
   bei System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   bei System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   bei System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   bei System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   bei System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   bei System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   bei System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   bei System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   bei Xunit.AppDomainManager_AppDomain.CreateObject[TObject](AssemblyName assemblyName, String typeName, Object[] args) in C:\Dev\xunit\xunit\src\xunit.runner.utility\AppDomain\AppDomainManager_AppDomain.cs:Zeile 78.
   bei Xunit.DiaSessionWrapper..ctor(String assemblyFilename) in C:\Dev\xunit\xunit\src\xunit.runner.utility\Utility\DiaSessionWrapper.cs:Zeile 22.
   bei Xunit.VisualStudioSourceInformationProvider..ctor(String assemblyFileName) in C:\Dev\xunit\xunit\src\xunit.runner.utility\Frameworks\VisualStudioSourceInformationProvider.cs:Zeile 24.
   bei Xunit.XunitFrontController..ctor(AppDomainSupport appDomainSupport, String assemblyFileName, String configFileName, Boolean shadowCopy, String shadowCopyFolder, ISourceInformationProvider sourceInformationProvider, IMessageSink diagnosticMessageSink) in C:\Dev\xunit\xunit\src\xunit.runner.utility\Frameworks\XunitFrontController.cs:Zeile 70.
   bei Xunit.Runner.VisualStudio.VsTestRunner.RunTestsInAssembly(IRunContext runContext, IFrameworkHandle frameworkHandle, LoggerHelper logger, TestPlatformContext testPlatformContext, RunSettings runSettings, IMessageSinkWithTypes reporterMessageHandler, AssemblyRunInfo runInfo) in C:\Dev\xunit\xunit\src\xunit.runner.visualstudio\VsTestRunner.cs:Zeile 495.

WRN: Protokollierung der Assemblybindung ist AUS.
Sie können die Protokollierung der Assemblybindungsfehler aktivieren, indem Sie den Registrierungswert [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) auf 1 festlegen.
Hinweis: Die Protokollierung der Assemblybindungsfehler führt zu einer gewissen Leistungseinbuße.
Sie können dieses Feature deaktivieren, indem Sie den Registrierungswert [HKLM\Software\Microsoft\Fusion!EnableLog] entfernen.

In #338. two workarounds are desribed:

a) Downgrade to 2.4.0
b) Add /p:Exclude="[xunit.*]*"

Workaround a) Downgrade to 2.4.0

When I downgrade, i get the following error:

The NUnit 3 driver encountered an error while executing reflected code.
   bei NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   bei NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   bei NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   bei NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   bei NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   bei NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   bei NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   bei NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   bei NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter)
   bei NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle)
Innerexception: System.TypeInitializationException: Der Typeninitialisierer für "Coverlet.Core.Instrumentation.Tracker.nunit.framework_9fb539db-510a-43a3-8fc0-8f135ed99489" hat eine Ausnahme verursacht. ---> System.Security.VerificationException: Dieser Vorgang kann die Laufzeit destabilisieren.
   bei Coverlet.Core.Instrumentation.Tracker.nunit.framework_9fb539db-510a-43a3-8fc0-8f135ed99489..cctor()
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei Coverlet.Core.Instrumentation.Tracker.nunit.framework_9fb539db-510a-43a3-8fc0-8f135ed99489.RecordHit(Int32 )
   bei NUnit.Framework.Api.FrameworkController..ctor(String assemblyNameOrPath, String idPrefix, IDictionary settings)
[xUnit.net 00:00:01.03] Skipping: Test (could not find dependent assembly 'nunit.framework, Version=3.12.0')
In C:\Users\dennis.boldt\git\smart-factory\zenon-addins\Build\Test.dll ist kein Test verfügbar. Stellen Sie sicher, dass Testdiscoverer und -executor registriert und die Versionseinstellungen für Plattform und Framework richtig sind, und versuchen Sie es anschließend noch mal.

Workaround b) Add /p:Exclude="[xunit.*]*"

When I add /p:Exclude="[xunit.*]*", I see some of the following warnings:

warning : [coverlet] Hits file: '[...]' not found for module: 'Utils' 

Additionally, I see a coverage report with 0%.

Install coverlet globally

dotnet tool install --global coverlet.console
coverlet Build\Test.dll --target "dotnet" --targetargs "test --no-build" --exclude "[xunit.*]*"

It shows again warning : [coverlet] Hits file: '[...]' not found for module: 'Utils'

Conclusion

I cannot get coverlet work. Any idea is appreciated.

@MarcoRossignoli
Copy link
Collaborator

MarcoRossignoli commented May 29, 2019

Can you try this #347?
exclude /p:Exclude="[xunit.*]*" with last version(no downgrade) <- sorry you've already done if I understood well.

@MarcoRossignoli MarcoRossignoli added the needs more info More details are needed label May 29, 2019
@boldt
Copy link
Author

boldt commented May 29, 2019

I just switched to 2.5.1 and it works with dotnet test --no-build /p:CollectCoverage=true now:

grafik

[Edit] Also coverlet standalone works with 2.5.1: coverlet Build\Test.dll --target "dotnet" --targetargs "test --no-build"

@MarcoRossignoli
Copy link
Collaborator

Ok, please can you try last version with verbose logging to understand what's happen? https://github.com/tonerdo/coverlet/blob/master/Documentation/Troubleshooting.md#msbuild-integration

@boldt
Copy link
Author

boldt commented May 29, 2019

To debug with msbuildlog, I swicthed back to 2.6.1 and it works now even without /p:Exclude="[xunit.*]*". I also do not have the "strong name"-error anymore and it prints out percentages (two digits instead of one):

grafik

No clue what's different. I still have the following warning:

warning : [coverlet] Hits file: '[...]' not found for module: 'Utils'

@MarcoRossignoli
Copy link
Collaborator

MarcoRossignoli commented May 29, 2019

Yes we rounded percentages #397
And for warning issue we fixed in master version #406 because is a false "negative" if module is not "tested"

You could try our nightly build to understand if it's fixed https://www.myget.org/feed/coverlet-dev/package/nuget/coverlet.msbuild/2.6.1-g0dd56242f2 (Wed, 29 May 2019 00:03:09 GMT)

@MarcoRossignoli
Copy link
Collaborator

You can fix verbosity with -v:q for msbuild(https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test?tabs=netcore21)
and with -v quiet for dotnet tool(console) https://github.com/tonerdo/coverlet#global-tool

@MarcoRossignoli
Copy link
Collaborator

Feel free to close this issue if your issue is resolved.

@MarcoRossignoli
Copy link
Collaborator

Ah for 0% coverage is possible you hit a know transient bug #110 we'll resolve ASAP with new collector features https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md (near to release).

@boldt
Copy link
Author

boldt commented May 29, 2019

I will check the Nightly on Monday.

@MarcoRossignoli
Copy link
Collaborator

Sure let us know!

@MarcoRossignoli
Copy link
Collaborator

@boldt we released new version https://www.nuget.org/packages/coverlet.msbuild/ let me know.

@boldt
Copy link
Author

boldt commented Jun 11, 2019

The warning is gone with 2.6.2. Thanks.

@boldt boldt closed this as completed Jun 11, 2019
russcam added a commit to russcam/apm-agent-dotnet that referenced this issue Jan 18, 2021
Without this causes a FileLoadException.
Looks to be related to coverlet-coverage/coverlet#449
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs more info More details are needed
Projects
None yet
Development

No branches or pull requests

2 participants