Add resource validation tests to prevent missing resource string errors

[Copilot]

• Create a test to verify all resource strings used in code exist in .resx files
  • Analyze the approach: scan assemblies for resource string references
  • Create test infrastructure to load and parse .resx files
  • Create test to extract resource references from source code using regex
  • Verify the test works on Microsoft.Build.dll (passing ✅)
  • Verify the test works on MSBuild.dll (passing ✅)
  • Address missing resources found by tests:
    • Tasks assembly: 12 missing resources in XamlTaskFactory (.NETFramework-only code) - documented with Skip
    • Utilities assembly: 1 missing resource (Message.InvalidImportance) - real bug documented with Skip
  • Add comprehensive documentation to test class
  • Add helper comments for extending tests to other assemblies
  • Run tests to ensure they work correctly (2 passing, 2 skipped)
  • Verify full build still works
  • Fix CS8604 nullable reference warning

Summary

Successfully implemented resource validation tests to prevent issues where code references resource strings that don't exist in .resx files.

What was added:

• New test file: src/Build.UnitTests/Resources_Tests.cs (275 lines)
• Test coverage: 4 test methods covering the main MSBuild assemblies
  • ✅ Microsoft.Build (passing)
  • ✅ MSBuild (passing)
  • ⏭️ Microsoft.Build.Tasks.Core (skipped - 12 missing resources documented)
  • ⏭️ Microsoft.Build.Utilities.Core (skipped - 1 missing resource documented)

How it works:

1. Parses .resx files to extract all available resource names
2. Scans C# source files using regex patterns to find resource string references
3. Validates that all referenced resources exist in the corresponding .resx files
4. Provides detailed output showing missing resources when validation fails

Benefits:

• Prevents runtime errors from missing resources
• Catches resources in wrong assemblies (cross-assembly references)
• Documents known issues for future resolution
• Easy to extend to additional assemblies
• Helps prevent issues like MSB4018: The "CreateItem" task failed unexpectedly, MSB0001: Internal MSBuild Error: Missing resource 'GlobExpansionFailed' #12334, Internal MSBuild Error: Missing resource 'RestoreComplete' on latest nightly sdk #11515, MissingRestorePropertyError doesn't exist #7218, Creating a Microsoft.Build.Evaluation.Project with an SDK fails with an "Internal MSBuild Error" #2997, [Bug]: TerminalLogger "on" and "off" are not recognized, results in resource errors. #9150

Recent fix:

• Fixed CS8604 nullable reference warning by adding null-forgiving operator

Original prompt

---

This section details on the original issue you should resolve

<issue_title>test that all used resources in code are present in resource files</issue_title>
<issue_description>Related to preventing issues like #12334 #11515 #7218 #2997 #9150

Most recently the issue was that the resource was accessible from multiple code paths (the class was compiled to 3 dlls) but tested only in one, leading to errors. A test case validating that resource files are matching what is used in a given dll would catch these.</issue_description>

<agent_instructions>please add test coverage based on the plan given by rainersigwald in the last comment</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@rainersigwald I think an analyzer would make a lot of sense--we have

msbuild/src/Shared/ResourceUtilities.cs

Lines 436 to 444 in e67b64e

	/// <summary>
	/// Verifies that a particular resource string actually exists in the string table. This will only be called in debug
	/// builds. It helps catch situations where a dev calls VerifyThrowXXX with a new resource string, but forgets to add the
	/// resource string to the string table, or misspells it!
	/// </summary>
	/// <remarks>This method is thread-safe.</remarks>
	/// <param name="resourceName">Resource string to check.</param>
	[Conditional("DEBUG")]
	internal static void VerifyResourceStringExists(string resourceName)

but it clearly hasn't done the trick--mostly because we don't have APIs to do all of the various conditions, so we have a lot of resource-logging calls that don't happen at all in normal operation.</comment_new>
<comment_new>@rainersigwald

But I'm not sure how to propagate the list from the analyser to the test — IIRC file I/O in analysers is not recommended.

Maybe generate a class per assembly that just has a list of the strings, internal to tests? Or pass the .resx files to the analyzer as additionalfiles and check directly?</comment_new>

Fixes #12413

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[dotnet-policy-service]

Hello @@copilot, I noticed that you’re changing an .swr file or any file under src/Package/MSBuild.VSSetup.. Please make sure to validate this change by an experimental VS insertion. This is accomplished by pushing to an exp/* branch, which requires write permissions to this repo.

[YuliiaKovalova]

@copilot resolve the issue
src\Build.UnitTests\Resources_Tests.cs(203,35): error CS8604: (NETCORE_ENGINEERING_TELEMETRY=Build) Possible null reference argument for parameter 'item' in 'bool HashSet.Add(string item)'.

[Copilot]

@copilot resolve the issue
src\Build.UnitTests\Resources_Tests.cs(203,35): error CS8604: (NETCORE_ENGINEERING_TELEMETRY=Build) Possible null reference argument for parameter 'item' in 'bool HashSet<s...

Fixed in commit a7a17ea by adding the null-forgiving operator (!) to line 203, which resolves the CS8604 warning while maintaining the null check on line 201.

[GangWang01]

The tests find the following referenced resource strings don't exist in the resource file.

Microsoft.Build.Tasks.Core

12 missing resource strings below only referenced in src\Tasks\XamlTaskFactory\TaskGenerator.cs don't exist in resource file of Microsoft.Build.Tasks.Core. It's even not able search them out in the whole repo including branches main, vs15.9 and vs16.4.

• AddDefaultsToActiveSwitchListDescription
• AddFallbacksToActiveSwitchListDescription
• AddValidateRelationsMethod
• ClassDescription
• ConstructorDescription
• EndSummary
• PropertyNameDescription
• PropertySwitchDescription
• PropertyTypeDescription
• StartSummary
• ToolExeFieldDescription
• ToolNameDescription

Microsoft.Build.Utilities.Core

1 resource string Message.InvalidImportance is not found in the resource file of Microsoft.Build.Utilities.Core, but it exists in the resource file of Microsoft.Build.Tasks.Core.
This resource string is used in the case invalid Importance is set to StandardErrorImportance/StandardOutputImportance of src\Utilities\ToolTask.cs. It's an important case. I added a new test to cover the case and it failed due to the missing resource string. We need to copy this resource string to the resource file of Microsoft.Build.Utilities.Core.

[rainersigwald]

Microsoft.Build.Tasks.Core

12 missing resource strings below only referenced in src\Tasks\XamlTaskFactory\TaskGenerator.cs don't exist in resource file of Microsoft.Build.Tasks.Core. It's even not able search them out in the whole repo including branches main, vs15.9 and vs16.4.

• AddDefaultsToActiveSwitchListDescription
• AddFallbacksToActiveSwitchListDescription
• AddValidateRelationsMethod
• ClassDescription
• ConstructorDescription
• EndSummary
• PropertyNameDescription
• PropertySwitchDescription
• PropertyTypeDescription
• StartSummary
• ToolExeFieldDescription
• ToolNameDescription

These . . . may never have been in the product? I can see the references added in MSBuild going back to dev10, but even in that old copy of the source I don't see a string.

The question now is "is this code reachable and has just been broken for this long?" or "does this reveal unreachable code we can delete?"

Microsoft.Build.Utilities.Core

1 resource string Message.InvalidImportance is not found in the resource file of Microsoft.Build.Utilities.Core, but it exists in the resource file of Microsoft.Build.Tasks.Core. This resource string is used in the case invalid Importance is set to StandardErrorImportance/StandardOutputImportance of src\Utilities\ToolTask.cs. It's an important case. I added a new test to cover the case and it failed due to the missing resource string. We need to copy this resource string to the resource file of Microsoft.Build.Utilities.Core.

Instead of duplicating it can we put it in src\Shared\Resources\Strings.shared.resx? That may already be in both assemblies.

[YuliiaKovalova]

Oh, i see what's going on with these resources in TaskGenerator
they all are guarded by GenerateComments property that is never ( in msbuild and azdo codebases) set to true.
I think it's safe to clean up the code here

[YuliiaKovalova]

Oh, i see what's going on with these resources in TaskGenerator they all are guarded by GenerateComments property that is never ( in msbuild and azdo codebases) set to true. I think it's safe to clean up the code here

@GangWang01 agreed with Rainer during standup that we can remove GenerateComments property and the code connected to it. Please do that.

[Copilot]

Pull request overview

This PR adds resource validation tests to prevent runtime errors caused by code referencing missing resource strings. The changes address several historical issues where resources were missing or in the wrong assembly.

Changes:

• Adds comprehensive resource validation test infrastructure in Resources_Tests.cs with regex-based extraction of resource references from source code
• Moves the Message.InvalidImportance resource from Tasks assembly to Shared resources (where it should be accessible by ToolTask)
• Removes unused GenerateComments functionality from XamlTaskFactory
• Adds a new test in ToolTask_Tests to verify invalid importance parameter handling

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/Build.UnitTests/Resources_Tests.cs	New test infrastructure to validate all referenced resources exist in .resx files
src/Shared/Resources/Strings.shared.resx	Adds Message.InvalidImportance resource to shared resources
src/Shared/Resources/xlf/*.xlf	Updates localization files with new Message.InvalidImportance entry
src/Tasks/Resources/Strings.resx	Removes Message.InvalidImportance (moved to shared) with comment explaining the move
src/Tasks/Resources/xlf/*.xlf	Removes Message.InvalidImportance from localized resources
src/Tasks/XamlTaskFactory/TaskGenerator.cs	Removes unused GenerateComments property and related code generation logic
src/Utilities.UnitTests/ToolTask_Tests.cs	Adds test for invalid importance parameter and updates MyTool constructor to accept ResourceManager