System.ArgumentException in SolutionState.CompilationTracker.cs

[mavasani]

Version Used: Latest roslyn master sources

Steps to Reproduce:

1. Sync to latest roslyn master, assuming repo root here is C:\roslyn.
2. Build Roslyn.sln in Debug configuration
3. Build AnalyzerRunner.csproj
4. Execute all IDE analyzers on Roslyn.sln using the following command from an admin command prompt: C:\roslyn\Binaries\Debug\Exes\AnalyzerRunner\AnalyzerRunner.exe C:\roslyn\Binaries\Debug\Exes\AnalyzerRunner c:\roslyn\Roslyn.sln /concurrent

Expected Behavior:
No exceptions

Actual Behavior:
System.ArgumentException: An item with the same key has already been added.
Also note that you might see assert from #31388

.Add invocation below seems to be throwing here when processing src\NuGet\VisualStudio\VS.ExternalAPIs.Roslyn.Package.csproj:

                                var metadataReference = await solution.GetMetadataReferenceAsync(
                                    projectReference, this.ProjectState, cancellationToken).ConfigureAwait(false);
 
                                // A reference can fail to be created if a skeleton assembly could not be constructed.
                                if (metadataReference != null)
                                {
                                    newReferences.Add(metadataReference);
                                    metadataReferenceToProjectId.Add(metadataReference, projectReference.ProjectId);
                                }
                                else
                                {
                                    hasSuccessfullyLoaded = false;
                                }

Call stack:

Failed to analyze VS.ExternalAPIs.Roslyn.Package with System.ArgumentException: An item with the same key has already been added.
   at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at Microsoft.CodeAnalysis.SolutionState.CompilationTracker.<FinalizeCompilationAsync>d__30.MoveNext() in C:\roslyn_2\src\Workspaces\Core\Portable\Workspace\Solution\SolutionState.CompilationTracker.cs:line 638
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.SolutionState.CompilationTracker.<BuildCompilationInfoFromScratchAsync>d__24.MoveNext() in C:\roslyn_2\src\Workspaces\Core\Portable\Workspace\Solution\SolutionState.CompilationTracker.cs:line 478
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.SolutionState.CompilationTracker.<GetOrBuildCompilationInfoAsync>d__22.MoveNext() in C:\roslyn_2\src\Workspaces\Core\Portable\Workspace\Solution\SolutionState.CompilationTracker.cs:line 407
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.SolutionState.CompilationTracker.<GetCompilationSlowAsync>d__19.MoveNext() in C:\roslyn_2\src\Workspaces\Core\Portable\Workspace\Solution\SolutionState.CompilationTracker.cs:line 316
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at AnalyzerRunner.Program.<GetProjectAnalysisResultAsync>d__10.MoveNext() in C:\roslyn_2\src\Tools\AnalyzerRunner\Program.cs:line 437

[mavasani]

Tagging @jasonmalinowski @heejaechang

[aolszowka]

We JUST Ran into this on one of our larger solution files (~1000 Projects; ~90,000 files) we'll try the patch and report back.

**Edit: So in researching this issue we found that we had a duplicated ProjectReference in one of our projects; removing this duplicate resolved the issue but it might be nice to get a little better error message. It appears that the suggested fix is just to ignore the duplicated Projects which seems wrong in my mind. For now we are going to fix our processes to avoid adding duplicate ProjectReferences.

[jasonmalinowski]

@aolszowka What tool were you using that was crashing? Visual Studio, or some command line tool that was processing your projects?

[aolszowka]

@jasonmalinowski Hi Jason, I was/am using using the StyleCopTester Tool (https://github.com/DotNetAnalyzers/StyleCopAnalyzers/tree/master/StyleCop.Analyzers/StyleCopTester) As I noted above I corrected the project that had the duplicate ProjectReference and have not had an issue since. I have also switched to generating Solution files to contain all of our projects (~3,000) that bad boy takes 25-30GB of RAM to run but man it fixes (most) of the errors :)

[jasonmalinowski]

@aolszowka OK, good, just wanted to confirm it was a tool that (behind the scenes) is using MSBuildWorkspace.

[jasonmalinowski]

Oops thought I had merged. My bad.