Source Generator generated files cause coverage generation error with source link + deterministic builds

[Romanx]

This is an odd combination of a few things but it seems that you get the following error when the following combination of things happens:

The Problem

If you have a project that uses a source generator (The attached example uses Generator.Equals for equality checks), uses sourcelink and deterministic builds then when running

dotnet test --collect:"XPlat Code Coverage" --settings ".\coverlet.runsettings"

you get

Data collector 'XPlat code coverage' message: [coverlet]Coverlet.Collector.Utilities.CoverletDataCollectorException: CoverletCoverageDataCollector: Failed to get coverage result
 ---> System.Collections.Generic.KeyNotFoundException: The given key '' was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key) in System.Private.CoreLib.dll:token 0x600675e+0x17
   at Coverlet.Core.Coverage.GetSourceLinkUrl(Dictionary`2 sourceLinkDocuments, String document) in /_/src/coverlet.core/Coverage.cs:line 510
   at Coverlet.Core.Coverage.CalculateCoverage() in /_/src/coverlet.core/Coverage.cs:line 391
   at Coverlet.Core.Coverage.GetCoverageResult() in /_/src/coverlet.core/Coverage.cs:line 173
   at Coverlet.Collector.DataCollection.CoverageWrapper.GetCoverageResult(Coverage coverage) in /_/src/coverlet.collector/DataCollection/CoverageWrapper.cs:line 52
   at Coverlet.Collector.DataCollection.CoverageManager.GetCoverageResult() in /_/src/coverlet.collector/DataCollection/CoverageManager.cs:line 93
   --- End of inner exception stack trace ---
   at Coverlet.Collector.DataCollection.CoverageManager.GetCoverageResult() in /_/src/coverlet.collector/DataCollection/CoverageManager.cs:line 98
   at Coverlet.Collector.DataCollection.CoverletCoverageCollector.OnSessionEnd(Object sender, SessionEndEventArgs e) in /_/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs:line 161.

Workaround

The workaround is that you can add an ExcludeByFile for *.g.cs which stops them hitting this case although there's obviously something unexpected happening.

Reproduction

CoverletRepoProject.zip

[MarcoRossignoli]

Thanks for reporting this

[AraHaan]

coverlet/src/coverlet.core/Helpers/InstrumentationHelper.cs

Lines 181 to 187 in 2bb04f2

	// We verify all docs and return false if not all are present in local
	// We could have false negative if doc is not a source
	// Btw check for all possible extension could be weak approach
	// We exlude from the check the autogenerated source file(i.e. source generators)
	// We exclude special F# construct https://github.com/coverlet-coverage/coverlet/issues/1145
	if (!_fileSystem.Exists(docName) && !docName.EndsWith(".g.cs") &&
	!IsUnknownModuleInFSharpAssembly(languageGuid, docName))

This needs also ignore .Designer.cs file due to an issue with the resx generator with coverlet.
To prevent it from ignoring winforms .Designer.cs files it could also check for the presence of // <auto-generated/>, // <autogenerated/>, and

// <autogenerated>
// a bit of text here that existed since the .NET Framework days...
// </autogenerated>

dotnet/roslyn-analyzers#5958

[sharwell]

Outputs from source generators are generally embedded in the PDB. Coverlet should always check the PDB for embedded source before checking for files on disk; when present, the embedded sources are always a more accurate source of truth than files on disk.

[MarcoRossignoli]

Thanks @sharwell for the info, we're also thinking to relax a bit the heuristic but maybe change a bit the order of the inspection could work for most of the cases

[petli]

It sounds like a good way to solve generated files once and for all. But are non-generated source files embedded in the PDB too? If they are, it would break the heuristic to automatically exclude third-party libraries completely.

[AraHaan]

If they do not exist in the repository, they are as far as source link is concerned (as long as they enable EmbedUntrackedSources or something if I remember right (which I always set to true by default as well as try to force all sources to be embedded even if they are in the repo).

[sharwell]

But are non-generated source files embedded in the PDB too?

It can be configured either way. You should be able to check the source location for the specific file and still determine its origin.

[Bertk]

Please verify whether the problem still exists using the latest coverlet 6.0.2 release.

[Romanx]

Tested this using the repo is 6.0.2 and the error didn't appear and the coverage file was generated. Seems fixed. I'll close this and open another with details if something similar occurs again