Enlighten RAR task by AR-May · Pull Request #13319 · dotnet/msbuild

AR-May · 2026-03-03T15:02:45Z

Context

RAR task enlightening. De-facto the only inputs that I saw unenlightened in the logs were StateFile, AppConfigFile and CandidateAssemblyFiles. However multiple other inputs, like TargetFrameworkDirectories or HintPath of assemblies are not guaranteed to be the full path. I added additional logic that ensures that the full path is used for them too.

Changes Made

RAR out-of-proc node now uses the multithreaded TaskEnvironment for proper path resolution (instead of just updating StateFile and AppConfigFile)
Moved Drivers to the Framework project - was required for above. It is in a separate PR Move task environment drivers to Framework. #13380.
Resolvers now take TaskEnvironement object and use them for path absolutisation of the inputs that are not absolutized prior.
Some of the inputs now are backed up with AbsolutePath structs instead of strings.

Breaking changes under a ChangeWave:

Empty AppConfigFile resulted before in failure while now it is skipped (as if it was null before)
StateFile.ItemSpec is used for the path instead of StateFile.ToString() which could be otherwise defined in a custom ITaskItem

Commits overview:

Make RAR as a service use TaskEnvironment: Updates out-of-process RAR client and supporting classes to be thread-safe and work through TaskEnvironment.
Absolutize RAR task inputs: Makes RAR inputs absolute paths and updates ReferenceTable to use TaskEnvironment instead of process-level environment.
Make resolvers use task environment: Updates all assembly resolvers (AssemblyFoldersResolver, GacResolver, HintPathResolver, etc.) to use TaskEnvironment instead of process environment.
Enlighten GetReferenceAssemblyPaths: Updates GetReferenceAssemblyPaths task and GlobalAssemblyCache to work with TaskEnvironment. This is required because GlobalAssemblyCache refactoring is needed for RAR task
Fix tests: Updates test files to work with the new TaskEnvironment-based RAR implementation, adding necessary setup and configuration.

Testing (in progress)

Checklist:

AR-May · 2026-03-04T13:48:27Z

src/Framework/FrameworkCommunicationsUtilities.cs

+        /// Returns key value pairs of environment variables in a dictionary
+        /// with a case-insensitive key comparer.
+        /// </summary>
+        internal static Dictionary<string, string> GetEnvironmentVariables()


TODO: Before we used functions from CommunicationUtilities that are more sophisticated and may have better perf. To avoid perf. regression we need to figure out a more proper migration of those functions from Shared files.

Copilot

Pull request overview

This PR advances “RAR task enlightening” by routing path resolution and environment access through TaskEnvironment (including in the out-of-proc RAR node), and by upgrading several RAR inputs from strings to AbsolutePath to ensure stable, project-relative-to-absolute resolution in multithreaded builds.

Changes:

RAR and related resolvers now use TaskEnvironment for absolute path conversion and environment variable access.
Out-of-proc RAR node now hydrates a multithreaded TaskEnvironment using the client project directory.
Introduces a new ChangeWave (18.6) and updates task/unit test code to provide TaskEnvironment.

Reviewed changes

Copilot reviewed 51 out of 51 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/Tasks/SystemState.cs	Uses `TaskEnvironment` for state file path handling during precomputed cache load.
src/Tasks/StateFileBase.cs	Clarifies expectation that state file path is absolute.
src/Tasks/RedistList.cs	Moves redist list paths to `AbsolutePath` and enables `TaskEnvironment`-based absolutization.
src/Tasks/InstalledSDKResolver.cs	Caches absolute SDK root paths for resolver probing.
src/Tasks/GetReferenceAssemblyPaths.cs	Marks task as multi-threadable and passes `TaskEnvironment` into GAC resolution.
src/Tasks/AssemblyDependency/Resolver.cs	Adds `TaskEnvironment` to resolver base for consistent path/env usage.
src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs	Converts several inputs to `AbsolutePath`, adds MT task interface, and updates RAR execution to pass `TaskEnvironment` through the pipeline.
src/Tasks/AssemblyDependency/ReferenceTable.cs	Propagates `TaskEnvironment` into resolution pipeline and normalizes resolved paths.
src/Tasks/AssemblyDependency/Reference.cs	Adds `IsFrameworkFile` overload for `AbsolutePath[]` framework paths.
src/Tasks/AssemblyDependency/RawFilenameResolver.cs	Resolves raw filename candidate via `TaskEnvironment`.
src/Tasks/AssemblyDependency/Node/RarNodeExecuteRequest.cs	Captures project directory for out-of-proc node `TaskEnvironment` creation.
src/Tasks/AssemblyDependency/Node/OutOfProcRarNodeEndpoint.cs	Creates a multithreaded `TaskEnvironment` per request using the client project directory.
src/Tasks/AssemblyDependency/HintPathResolver.cs	Resolves hint path using `TaskEnvironment` and canonicalizes.
src/Tasks/AssemblyDependency/GlobalAssemblyCache.cs	Switches env-var access to `TaskEnvironment` and threads it through GetLocation APIs.
src/Tasks/AssemblyDependency/GacResolver.cs	Threads `TaskEnvironment` into GAC resolver construction.
src/Tasks/AssemblyDependency/FrameworkPathResolver.cs	Threads `TaskEnvironment` into framework path resolver construction.
src/Tasks/AssemblyDependency/DirectoryResolver.cs	Caches absolute search path via `TaskEnvironment`.
src/Tasks/AssemblyDependency/CandidateAssemblyFilesResolver.cs	Threads `TaskEnvironment` into candidate assembly files resolver construction.
src/Tasks/AssemblyDependency/AssemblyResolution.cs	Threads `TaskEnvironment` through resolver compilation APIs.
src/Tasks/AssemblyDependency/AssemblyFoldersResolver.cs	Threads `TaskEnvironment` into AssemblyFolders resolver.
src/Tasks/AssemblyDependency/AssemblyFoldersFromConfig/AssemblyFoldersFromConfigResolver.cs	Uses `TaskEnvironment` for cache escape-hatch env var access and passes it into cache.
src/Tasks/AssemblyDependency/AssemblyFoldersFromConfig/AssemblyFoldersFromConfigCache.cs	Uses `TaskEnvironment` for cache escape-hatch env var access.
src/Tasks/AssemblyDependency/AssemblyFoldersExResolver.cs	Uses `TaskEnvironment` for cache escape-hatch env var access and passes it into cache.
src/Tasks.UnitTests/RARPrecomputedCache_Tests.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/HintPathResolver_Tests.cs	Updates resolver test to provide `TaskEnvironment`.
src/Tasks.UnitTests/GetReferencePaths_Tests.cs	Updates task test to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/WinMDTests.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/VerifyTargetFrameworkHigherThanRedist.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/VerifyTargetFrameworkAttribute.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/VerifyIgnoreVersionForFrameworkReference.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/SuggestedRedirects.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/StronglyNamedDependencyAutoUnify.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/StronglyNamedDependencyAppConfig.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/StronglyNamedDependency.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/SpecificVersionPrimary.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/ResolveAssemblyReferenceTestFixture.cs	Updates helper to pass `TaskEnvironment` into GAC resolution.
src/Tasks.UnitTests/AssemblyDependency/Perf.cs	Updates perf tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/NonSpecificVersionStrictPrimary.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/Node/RarNodeExecuteRequest_Tests.cs	Updates node request tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/Node/OutOfProcRarNode_Tests.cs	Updates node tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/Miscellaneous.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/InstalledSDKResolverFixture.cs	Updates resolver fixture to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/GlobalAssemblyCacheTests.cs	Updates cache tests to pass `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/FilePrimary.cs	Updates RAR tests to provide `TaskEnvironment`.
src/Tasks.UnitTests/AssemblyDependency/AssemblyFoldersFromConfig_Tests.cs	Updates tests to provide `TaskEnvironment`.
src/Framework/MultiThreadedTaskEnvironmentDriver.cs	Moves driver into Framework layer and updates env-var comparer dependency.
src/Framework/MultiProcessTaskEnvironmentDriver.cs	Moves driver into Framework layer and routes env-var ops through Framework utilities.
src/Framework/FrameworkCommunicationsUtilities.cs	Adds Framework-local env-var utilities needed by moved drivers.
src/Framework/ChangeWaves.cs	Adds new wave 18.6 and includes it in `AllWaves`.
src/Build/Microsoft.Build.csproj	Removes compilation of drivers from Build project (now in Framework).
src/Build/BackEnd/Components/RequestBuilder/RequestBuilder.cs	Updates comment to reflect new Framework env-var utilities usage.

Comments suppressed due to low confidence (1)

src/Tasks/AssemblyDependency/ReferenceTable.cs:496

When assemblyFileName is relative, reference.FullPath is made absolute via TaskEnvironment, but the subsequent _fileExists, _directoryExists, _getAssemblyName, and ResolvedSearchPath logic still uses the original (potentially relative) assemblyFileName. In multithreaded/out-of-proc scenarios this can probe the wrong directory and fail to resolve assemblies. Use the resolved absolute path (e.g., reference.FullPath) consistently for file system checks and assembly name probing.

src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs

src/Tasks/SystemState.cs

src/Tasks/AssemblyDependency/RawFilenameResolver.cs

Copilot · 2026-03-04T13:57:05Z

src/Tasks/InstalledSDKResolver.cs

            _resolvedSDKs = resolvedSDKs;
+            _resolvedSDKPaths = new Dictionary<string, string>(resolvedSDKs.Count);
+
+            // Cache absolute paths to avoid repeated TaskEnvironment.GetAbsolutePath calls
+            foreach (var kvp in resolvedSDKs)
+            {
+                _resolvedSDKPaths[kvp.Key] = taskEnvironment.GetAbsolutePath(kvp.Value.ItemSpec);
+            }


_resolvedSDKPaths is created without a case-insensitive comparer. If sdkName differs in casing from the keys stored in _resolvedSDKs (which is OrdinalIgnoreCase), _resolvedSDKs.TryGetValue can succeed while _resolvedSDKPaths.TryGetValue fails, causing the SDK resolver to incorrectly skip resolution. Construct _resolvedSDKPaths with StringComparer.OrdinalIgnoreCase (or reuse resolvedSDKs.Comparer) to match _resolvedSDKs behavior.

src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs

src/Tasks/AssemblyDependency/AssemblyFoldersFromConfig/AssemblyFoldersFromConfigResolver.cs

src/Tasks/AssemblyDependency/ReferenceTable.cs

src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs

src/Build/Microsoft.Build.csproj

src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs

JanProvaznik · 2026-03-06T11:19:56Z

src/Tasks/StateFileBase.cs


        /// <summary>
        /// Reads the specified file from disk into a StateFileBase derived object.
+        /// stateFile should be absolute path to the file on disk.


can it be AbsolutePath type?

I can look into it. In some cases, I decided to not do that because I will need to jump back and forth between the AbsolutePath and string. This one probably should be possible to just switch to AbsolutePath.

Ok, I did not do that because there are other tasks that use it. However, I can create a copy of this function that would take AbsolutePath and once all other tasks are enlightened, we can remove the old function.

JanProvaznik · 2026-03-06T11:28:41Z

src/Tasks/AssemblyDependency/AssemblyFoldersResolver.cs

            if (assemblyName != null)
            {
-                // {AssemblyFolders} was passed in.
+                // {AssemblyFolders} was passed in. They come from registry and are assumed to be absolute paths.


isn't that the utility of AbsolutePath? we can just wrap each of these to get this guarantee on the code level

I guess we can do that to additionally ensure it!

Ah, looked again and remembered why i did not want to do that. I do not want to cast all the time in the resolver, but I did not want to do this on the source either - as the values are computed once and then stored in the static cache, same situation as in ToolLocationHelper.

src/Tasks/AssemblyDependency/ReferenceTable.cs

JanProvaznik · 2026-03-06T11:35:10Z

src/Tasks.UnitTests/AssemblyDependency/AssemblyFoldersFromConfig_Tests.cs

                ResolveAssemblyReference t = new ResolveAssemblyReference
                {
                    BuildEngine = new MockEngine(_output),
+                    TaskEnvironment = TaskEnvironmentHelper.CreateForTest(),


we were quite lazy about this test desgin in prior prs., should we consider being less lazy and having both taskenvironments in each test? or are we confident they work the same way from unit testing them?

This env is always multiprocess, which is great for catching regression, but not asserting correctness of mt.

Yes, absolutely, I had exactly same thoughts about it! Yes, we should and I was looking into doing that in a separate PR - for every test that uses TaskEnvironmentHelper.CreateForTest()

Idea for MT would be to set the mt environment and then move the current directory to something else. In the end of the test, maybe on the dispose to ensure it, we would return it back. And for MP environment it would be same as now.

…read safe.

…nment.

AR-May · 2026-03-13T12:33:42Z

I separated moving drivers to the Framework project to another PR: #13380.
Merge this changes after #13380 is merged. There would be build errors meanwhile.

SimaTian · 2026-03-24T13:07:43Z

src/Tasks.UnitTests/AssemblyDependency/GlobalAssemblyCacheTests.cs

            ResolveAssemblyReference t = new ResolveAssemblyReference();

            t.BuildEngine = new MockEngine(_output);
+            t.TaskEnvironment = TaskEnvironmentHelper.CreateForTest();


Just a general question - we've got design discussion about a default for the TaskEnvironment. If/when it goes through, can we then remove all these? It's a lot of noise that does basically nothing.

yes, though it'd be super nice to test with both test environments which we now don't do :(

SimaTian · 2026-03-24T13:14:41Z

src/Tasks/AssemblyDependency/Node/OutOfProcRarNodeEndpoint.cs

+
+                            // TODO: This needs to be updated when configuration support is added to pass client-specific state
+                            // For now, using the environment variables from the RAR node process.
+                            using (var environmentDriver = new MultiThreadedTaskEnvironmentDriver(request.ProjectDirectory))


Are these failures related to the defaults we need or are these a different beast altogether? (seems like this will need a similar stub like the one I created for the SDK)

SimaTian · 2026-03-24T13:18:08Z

src/Tasks/AssemblyDependency/AssemblyFoldersFromConfig/AssemblyFoldersFromConfigCache.cs

            else
            {
                _filesInDirectories = new(assemblyFoldersFromConfig.AsParallel()
                    .Where(assemblyFolder => FileUtilities.DirectoryExistsNoThrow(assemblyFolder.DirectoryPath))


Do we have any guarantee this path will be properly absolutized?

According to copilot:

_filesInDirectories cache is broken if paths aren't absolute. Directory.GetFiles(relativeDir) stores relative strings in the HashSet, but Contains(path) receives paths from ResolveFromDirectory which may be in a different form → silent cache miss.

SimaTian · 2026-03-24T13:19:12Z

src/Tasks/AssemblyDependency/AssemblyFoldersFromConfig/AssemblyFoldersFromConfigResolver.cs

                                }

                                // Lets see if the processor architecture matches, note this this method will cache the result when it was first called.
                                AssemblyNameExtension foundAssembly = getAssemblyName(candidatePath);


do we have any guarantee that this path will be properly absolutized?

I'm not saying it isn't - didn't look that deep. However if the code changes elsewhere, we have no visibility here.

AR-May self-assigned this Mar 4, 2026

AR-May commented Mar 4, 2026

View reviewed changes

AR-May marked this pull request as ready for review March 4, 2026 13:48

Copilot AI review requested due to automatic review settings March 4, 2026 13:48

Copilot started reviewing on behalf of AR-May March 4, 2026 13:49 View session

Copilot AI reviewed Mar 4, 2026

View reviewed changes

JanProvaznik reviewed Mar 6, 2026

View reviewed changes

AR-May added 5 commits March 13, 2026 13:04

Make RAR as a service use TaskEnvironment. Make OutOfProcRarClient th…

b743044

…read safe.

Absolutize RAR task inputs. ReferenceTable should use the task enviro…

f5caaff

…nment.

Make resolvers use task environemnt.

f090cef

Enlighten GetReferenceAssemblyPaths

84ed710

Fix tests.

fdd2f52

AR-May force-pushed the enlighten-RAR-2 branch from 2c6a029 to fdd2f52 Compare March 13, 2026 12:31

Set default driver to enlightened tasks.

b669d97

SimaTian reviewed Mar 24, 2026

View reviewed changes

Conversation

AR-May commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Changes Made

Commits overview:

Testing (in progress)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AR-May commented Mar 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SimaTian Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AR-May commented Mar 3, 2026 •

edited

Loading

SimaTian Mar 24, 2026 •

edited

Loading