Improve source indexer deduplication to prefer real implementation over *.notsupported.cs files

.github/workflows/dotnet.yml

@@ -1,21 +1,25 @@
-name: .NET
-
-on:
-  push:
-    branches: [ main ]
-  pull_request:
-    branches: [ main ]
-
-jobs:
-  build:
-    runs-on: windows-latest
-    steps:
-    - uses: actions/checkout@v4
-    - name: Setup .NET
-      uses: actions/setup-dotnet@v4
-      with:
-        global-json-file: global.json
-    - name: Restore source-indexer.sln
-      run: dotnet restore src/source-indexer.sln
-    - name: Build source-indexer.sln
-      run: dotnet build --no-restore src/source-indexer.sln
+name: .NET
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  build:
+    runs-on: windows-latest
+    steps:
+    - uses: actions/checkout@v4
+    - name: Setup .NET
+      uses: actions/setup-dotnet@v4
+      with:
+        global-json-file: global.json
+    - name: Restore source-indexer.sln
+      run: dotnet restore src/source-indexer.sln
+    - name: Restore SourceBrowser.sln
+      run: dotnet restore src/SourceBrowser/SourceBrowser.sln
+    - name: Build & test source-indexer.sln
+      run: dotnet test --no-restore src/source-indexer.sln
+    - name: Build & test SourceBrowser.sln
+      run: dotnet test --no-restore src/SourceBrowser/SourceBrowser.sln

README.md

@@ -1,6 +1,9 @@
 # source-indexer
 This repo contains the code for building http://source.dot.net
 
+## Documentation
+- [Source Selection Algorithm](docs/source-selection-algorithm.md) - How the indexer chooses the best implementation when multiple builds exist for the same assembly
+
 ## Build Status
 [![Build Status](https://dev.azure.com/dnceng/internal/_apis/build/status/dotnet-source-indexer/dotnet-source-indexer%20CI?branchName=main)](https://dev.azure.com/dnceng/internal/_build/latest?definitionId=612&branchName=main)
 

docs/source-selection-algorithm.md

@@ -0,0 +1,75 @@
+# Source Selection Algorithm
+
+When the source indexer processes multiple builds for the same assembly (e.g., generic builds, platform-specific builds, or builds with different target frameworks), it uses a scoring algorithm to select the "best" implementation to include in the final source index.
+
+## Overview
+
+The deduplication process groups all compiler invocations by `AssemblyName` and then calculates a score for each build. The build with the highest score is selected and included in the generated solution file.
+
+## Scoring Priorities
+
+The scoring algorithm evaluates builds using the following criteria, ordered by priority from highest to lowest:
+
+### 1. UseForSourceIndex Property (Highest Priority)
+- **Score**: `int.MaxValue` (2,147,483,647)
+- **Description**: When a project explicitly sets the `UseForSourceIndex` property to `true`, it receives the maximum possible score, ensuring it will always be selected regardless of other factors.
+- **Use Case**: Provides an escape hatch for projects that should definitely be included in the source index.
+
+### 2. Platform Support Status (Second Priority)
+- **Score**: `-10,000` penalty for platform-not-supported assemblies
+- **Description**: If a project has the `IsPlatformNotSupportedAssembly` property set to `true`, it receives a heavy penalty.
+- **Use Case**: Ensures that stub implementations containing mostly `PlatformNotSupportedException` are avoided in favor of real implementations.
+
+### 3. Target Framework Version (Third Priority)
+- **Score**: `Major * 1000 + Minor * 100`
+- **Description**: Newer framework versions receive higher scores. For example:
+  - .NET 8.0 = 8,000 + 0 = 8,000 points
+  - .NET 6.0 = 6,000 + 0 = 6,000 points
+  - .NET Framework 4.8 = 4,000 + 80 = 4,080 points
+- **Use Case**: Prefers more recent implementations that are likely to contain the latest features and bug fixes.
+
+### 4. Platform Specificity (Fourth Priority)
+- **Score**: `+500` for platform-specific frameworks
+- **Additional**: `+100` bonus for Linux platforms, `+50` bonus for Unix platforms
+- **Description**: Platform-specific builds (e.g., `net8.0-linux`, `net8.0-windows`) receive bonuses over generic builds.
+- **Use Case**: Platform-specific implementations often contain more complete functionality than generic implementations.
+
+### 5. Source File Count (Lowest Priority)
+- **Score**: `+1` per source file
+- **Description**: Builds with more source files receive higher scores.
+- **Use Case**: Acts as a tiebreaker when other factors are equal, assuming more source files indicate a more complete implementation.
+
+## Example Scoring
+
+Consider these hypothetical builds for `System.Net.NameResolution`:
+
+| Build | UseForSourceIndex | IsPlatformNotSupported | Framework | Platform | Source Files | Total Score |
+|-------|-------------------|------------------------|-----------|----------|--------------|-------------|
+| Generic Build | false | true | net8.0 | none | 45 | -1,955 |
+| Linux Build | false | false | net8.0-linux | linux | 127 | 8,727 |
+| Windows Build | false | false | net8.0-windows | windows | 98 | 8,598 |
+| Override Build | true | false | net6.0 | none | 23 | 2,147,483,647 |
+
+In this example:
+- The **Override Build** would be selected due to `UseForSourceIndex=true`
+- Without the override, the **Linux Build** would be selected with the highest score
+- The **Generic Build** receives a massive penalty for being platform-not-supported
+
+## Implementation Details
+
+The scoring logic is implemented in the `CalculateInvocationScore` method in `BinLogToSln/Program.cs`. The method:
+
+1. Reads project properties from the binlog file
+2. Applies scoring rules in priority order
+3. Handles parsing errors gracefully
+4. Returns a base score of 1 for builds that fail scoring to avoid complete exclusion
+
+## Configuration
+
+The algorithm can be influenced through MSBuild project properties:
+
+- **UseForSourceIndex**: Set to `true` to force selection of this build
+- **IsPlatformNotSupportedAssembly**: Set to `true` to indicate this is a stub implementation
+- **TargetFramework**: Automatically detected from the project file
+
+These properties are captured from the binlog during the build analysis phase.

src/SourceBrowser/SourceBrowser.sln

@@ -19,7 +19,9 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "SourceIndexServer.Tests", "
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "BinLogParser", "src\BinLogParser\BinLogParser.csproj", "{4EF5052C-7D88-49C6-B940-5190CECD070D}"
 EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "BinLogToSln", "src\BinLogToSln\BinLogToSln.csproj", "{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}"
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "BinLogToSln", "src\BinLogToSln\BinLogToSln.csproj", "{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}"
+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "BinLogToSln.Tests", "src\BinLogToSln.Tests\BinLogToSln.Tests.csproj", "{A6F4AA1E-2B2A-4E48-9C3E-4A1B2D3C5E7F}"
 EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{C0B9CC1C-1EF1-4086-9532-E8679CBA4E62}"
 	ProjectSection(SolutionItems) = preProject
@@ -65,10 +67,14 @@ Global
 		{4EF5052C-7D88-49C6-B940-5190CECD070D}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{4EF5052C-7D88-49C6-B940-5190CECD070D}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{4EF5052C-7D88-49C6-B940-5190CECD070D}.Release|Any CPU.Build.0 = Release|Any CPU
-		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
-		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Debug|Any CPU.Build.0 = Debug|Any CPU
-		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Release|Any CPU.ActiveCfg = Release|Any CPU
-		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Release|Any CPU.Build.0 = Release|Any CPU
+		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{E73D1784-F1BC-4F01-B68E-03623CFBFB8E}.Release|Any CPU.Build.0 = Release|Any CPU
+		{A6F4AA1E-2B2A-4E48-9C3E-4A1B2D3C5E7F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{A6F4AA1E-2B2A-4E48-9C3E-4A1B2D3C5E7F}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{A6F4AA1E-2B2A-4E48-9C3E-4A1B2D3C5E7F}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{A6F4AA1E-2B2A-4E48-9C3E-4A1B2D3C5E7F}.Release|Any CPU.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE