Add support for fallback search paths for Target frameworks (dotnet#60)

* Revert "[RFC]: Add support for fallback search paths for Target frameworks"

This reverts commit 0826df1.

* GetReferenceAssemblyPaths: Add `TargetFrameworkFallbackSearchPaths`

.. property. This specifies a list of ';' separated paths which are used
as fallback search paths, for looking up the target framework, if could
not be found in @rootpath.

* Add $(TargetFrameworkFallbackSearchPaths) for looking up target

.. frameworks. This contains ';' separated list of paths. When
`GetReferenceAssemblyPaths` task will look for assemblies corresponding to a
target framework, then it will try to find it in multiple locations:

	1. $(TargetFrameworkRootPath) or the default path if
	$(TargetFrameworkRootPath) is ''

	2. Each path in $(TargetFrameworkFallbackSearchPaths)

* [tests] ToolLocationHelper_Tests: Move to use TestEnvironment

.. in preparation for upcoming commits.

* ToolLocationHelper APIs: Add support for target framework fallback ..

.. search paths.

`ToolLocationHelper.GetPathTo{StandardLibraries,ReferenceAssemblies}`
look for the target framework in the path specified by
@targetFrameworkRootPath parameter (roughly corresponding to the
$(TargetFrameworkRootPath) property) or the default path.

It would be useful to be able to look for these frameworks in more than
one location. For example, in case of Xamarin products on OSX, mobile
specific frameworks are installed in
`/Library/Frameworks/Mono.framework/External/xbuild-frameworks` and the
.NET ones are installed in
`/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/xbuild-frameworks` .

With support for fallback search paths, a lookup for a target framework will follow the order:

		1. @targetFrameworkRootPath or default location if @targetFrameworkRootPath is ''
		2. Fallback search paths

APIs changed:

GetPathToStandardLibraries(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string platformTarget, string targetFrameworkRootPath)
	changed to => GetPathToStandardLibraries(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string platformTarget, string targetFrameworkRootPath, string targetFrameworkFallbackSearchPaths = null)

GetPathToReferenceAssemblies(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string platformTarget, string targetFrameworkRootPath)
	changed to => GetPathToReferenceAssemblies(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string platformTarget, string targetFrameworkRootPath, string targetFrameworkFallbackSearchPaths = null)

New overload:

GetPathToReferenceAssemblies(string targetFrameworkRootPath, string targetFrameworkFallbackSearchPaths, FrameworkNameVersioning frameworkName)

- The reason for using a `targetFrameworkRootPath` parameter separate from
the `targetFrameworkFallbackSearchPaths` is that we need to
automatically fallback to a default path, which is being determined in
code (`FrameworkLocationHelper.programFilesReferenceAssemblyLocation`).

- Also, the parameter `targetFrameworkRootPath` roughly corresponds to the
`$(TargetFrameworkRootPath)` property and changing that to have multiple
paths would change it's meaning.

* [mono] Set default $(TargetFrameworkFallbackSearchPaths) for osx

- This sets up the $(TargetFrameworkFallbackSearchPaths) to point to
  `/Library/Frameworks/Mono.framework/External/xbuild-frameworks` on
  mono/osx. This location is used by Xamarin to install frameworks.
  The .net frameworks are available in usual default location.

  So, effectively, target frameworks are looked up, in order:

  1. $(TargetFrameworkRootPath) or the default location, if
	 $(TargetFrameworkRootPath) is ''

  2. If not found in (1), then try to find it in the fallback search
	 paths specified in $(TargetFrameworkFallbackSearchPaths)

Note that we always add the `External` as the last fallback path on
`$(TargetFrameworkFallbackSearchPaths)`.

Rationale:

When running on Mono, target frameworks are searched for in
`$mono_prefix/lib/mono/xbuild-frameworks`, which becomes
`/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/xbuild-frameworks`.

This works, but has a problem when upgrading the Mono.framework package:
a *new* `/Library/Frameworks/Mono.framework/Versions/@Version@` directory
is created, so anything installed into the previous directory will be
"lost". The file is still there, but xbuild will no longer find it.

To address this upgrade scenario, Mono's xbuild also checks a fallback path:

	`/Library/Frameworks/Mono.framework/External/xbuild-frameworks`

This location is not within a versioned framework directory, and thus files
installed into it will survive Mono.framework package upgrades.

This also allows overriding the root path to test in-tree builds of custom target
frameworks. Simply changing `$(TargetFrameworkRootPath)` would mean that
the in-tree builds of the custom frameworks would work but the default
ones would not be found!

mono/ExtensionsPath-ToolsVersion/Microsoft.Common.targets/ImportBefore/Microsoft.Common.Mono.Before.targets

@@ -24,6 +24,8 @@ Copyright (C) Microsoft Corporation. All rights reserved.
 
         <!-- Disable generation of serialization assemblies for now. workaround for bxc #55697 -->
         <GenerateSerializationAssemblies Condition="'$(GenerateSerializationAssemblies)' == ''">Off</GenerateSerializationAssemblies>
+
+        <TargetFrameworkFallbackSearchPaths Condition="$([MSBuild]::IsOSPlatform('osx'))">$(TargetFrameworkFallbackSearchPaths);/Library/Frameworks/Mono.framework/External/xbuild-frameworks</TargetFrameworkFallbackSearchPaths>
     </PropertyGroup>
 
 </Project>

ref/Microsoft.Build.Utilities.Core/net/Microsoft.Build.Utilities.Core.cs

@@ -458,11 +458,14 @@ public static partial class ToolLocationHelper
         public static string GetPathToDotNetFrameworkSdkFile(string fileName, Microsoft.Build.Utilities.TargetDotNetFrameworkVersion version, Microsoft.Build.Utilities.VisualStudioVersion visualStudioVersion, Microsoft.Build.Utilities.DotNetFrameworkArchitecture architecture) { throw null; }
         public static System.Collections.Generic.IList<string> GetPathToReferenceAssemblies(System.Runtime.Versioning.FrameworkName frameworkName) { throw null; }
         public static System.Collections.Generic.IList<string> GetPathToReferenceAssemblies(string targetFrameworkRootPath, System.Runtime.Versioning.FrameworkName frameworkName) { throw null; }
+        public static System.Collections.Generic.IList<string> GetPathToReferenceAssemblies(string targetFrameworkRootPath, string targetFrameworkFallbackSearchPaths, System.Runtime.Versioning.FrameworkName frameworkName) { throw null; }
         public static System.Collections.Generic.IList<string> GetPathToReferenceAssemblies(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile) { throw null; }
         public static System.Collections.Generic.IList<string> GetPathToReferenceAssemblies(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string targetFrameworkRootPath) { throw null; }
+        public static System.Collections.Generic.IList<string> GetPathToReferenceAssemblies(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string targetFrameworkRootPath, string targetFrameworkFallbackSearchPaths=null) { throw null; }
         public static string GetPathToStandardLibraries(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile) { throw null; }
         public static string GetPathToStandardLibraries(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string platformTarget) { throw null; }
         public static string GetPathToStandardLibraries(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string platformTarget, string targetFrameworkRootPath) { throw null; }
+        public static string GetPathToStandardLibraries(string targetFrameworkIdentifier, string targetFrameworkVersion, string targetFrameworkProfile, string platformTarget, string targetFrameworkRootPath, string targetFrameworkFallbackSearchPaths=null) { throw null; }
         public static string GetPathToSystemFile(string fileName) { throw null; }
         [System.ObsoleteAttribute("Consider using GetPlatformSDKLocation instead")]
         public static string GetPathToWindowsSdk(Microsoft.Build.Utilities.TargetDotNetFrameworkVersion version, Microsoft.Build.Utilities.VisualStudioVersion visualStudioVersion) { throw null; }

src/MSBuild/app.config

@@ -88,7 +88,6 @@
         <property name="MSBuildRuntimeVersion" value="4.0.30319" />
         <property name="VisualStudioVersion" value="15.0" />
 
-        <property name="TargetFrameworkRootPathSearchPathsOSX" value="/Library/Frameworks/Mono.framework/External/xbuild-frameworks/" />
         <property name="RoslynTargetsPath" value="$([MSBuild]::GetToolsDirectory32())\Roslyn" />
 
         <projectImportSearchPaths>

src/Shared/FrameworkLocationHelper.cs

@@ -1002,85 +1002,6 @@ private static string GetPathToBuildToolsFromEnvironment(DotNetFrameworkArchitec
             }
         }
 
-        internal static IList<string> GetTargetFrameworkRootFallbackPaths(string toolsVersion)
-        {
-#if FEATURE_SYSTEM_CONFIGURATION
-            return GetTargetFrameworkRootFallbackPathsFromConfigFor(toolsVersion);
-#else
-            return new List<string>();
-#endif
-        }
-
-#if FEATURE_SYSTEM_CONFIGURATION
-        /// <summary>
-        /// Returns the list of fallback search paths for looking up Target frameworks for the current OS,
-        /// specified in app.config like:
-        ///
-        ///
-        /// </summary>
-        internal static IList<string> GetTargetFrameworkRootFallbackPathsFromConfigFor(string toolsVersion)
-        {
-            try
-            {
-                ToolsetElement toolset = GetToolsetElementFromConfigFor(toolsVersion);
-                PropertyElement searchPathsfromConfiguration = toolset?.PropertyElements.GetElement("TargetFrameworkRootPathSearchPaths" + GetOSNameForTargetFrameworkRoot());
-                var searchPaths = searchPathsfromConfiguration?.Value;
-
-                if (searchPaths != null)
-                {
-                    //FIXME: Split on unix
-                    var pathsList = searchPaths.Split(new char[] {';'}, StringSplitOptions.RemoveEmptyEntries);
-                    return pathsList;
-                }
-            }
-            catch (ConfigurationException)
-            {
-                // may happen if the .exe.config contains bad data.  Shouldn't ever happen in 
-                // practice since we'll long since have loaded all toolsets in the toolset loading 
-                // code and thrown errors to the user at that point if anything was invalid, but just 
-                // in case, just eat the exception here, so that we can go on to look in the registry
-                // to see if there is any valid data there.  
-            }
-            return new List<string>();
-        }
-#endif
-
-#if FEATURE_SYSTEM_CONFIGURATION
-        private static ToolsetElement GetToolsetElementFromConfigFor(string toolsVersion)
-        {
-            ToolsetConfigurationSection configurationSection = null;
-
-            if (ToolsetConfigurationReaderHelpers.ConfigurationFileMayHaveToolsets())
-            {
-                Configuration configuration = ConfigurationManager.OpenExeConfiguration(
-                                                BuildEnvironmentHelper.Instance.CurrentMSBuildExePath);
-
-                configurationSection = ToolsetConfigurationReaderHelpers.ReadToolsetConfigurationSection(configuration);
-            }
-
-            return configurationSection?.Toolsets.GetElement(toolsVersion);
-        }
-#endif
-
-        /// <summary>
-        /// OS name that can be used as the suffix for `TargetFrameworkRootPathSearchPaths` property name
-        /// in app.config
-        /// </summary>
-        private static string GetOSNameForTargetFrameworkRoot()
-        {
-            if (NativeMethodsShared.IsWindows)
-            {
-                return "Windows";
-            }
-
-            if (NativeMethodsShared.IsOSX)
-            {
-                return "OSX";
-            }
-
-            return "Unix";
-        }
-
 #if FEATURE_WIN32_REGISTRY
         /// <summary>
         /// Look up the path to the build tools directory in the registry for the requested ToolsVersion and requested architecture  

src/Tasks.UnitTests/GetReferencePaths_Tests.cs

@@ -294,6 +294,45 @@ public void TestGeneralFrameworkMonikerGoodWithInvalidCharInIncludePath()
                 }
             }
         }
+
+        /// <summary>
+        /// Test the case where there is a good target framework moniker passed in.
+        /// </summary>
+        [Fact]
+        public void TestGeneralFrameworkMonikerGoodWithFrameworkInFallbackPaths()
+        {
+            using (var env = TestEnvironment.Create())
+            {
+                string frameworkRootDir = Path.Combine(env.DefaultTestDirectory.FolderPath, "framework-root");
+                var framework41Directory = env.CreateFolder(Path.Combine(frameworkRootDir, Path.Combine("MyFramework", "v4.1") + Path.DirectorySeparatorChar));
+                var redistListDirectory = env.CreateFolder(Path.Combine(framework41Directory.FolderPath, "RedistList"));
+
+                string redistListContents =
+                        "<FileList Redist='Microsoft-Windows-CLRCoreComp' Name='.NET Framework 4.1'>" +
+                            "<File AssemblyName='System.Xml' Version='2.0.0.0' PublicKeyToken='b03f5f7f11d50a3a' Culture='Neutral' FileVersion='2.0.50727.208' InGAC='true' />" +
+                             "<File AssemblyName='Microsoft.Build.Engine' Version='2.0.0.0' PublicKeyToken='b03f5f7f11d50a3a' Culture='Neutral' FileVersion='2.0.50727.208' InGAC='true' />" +
+                        "</FileList >";
+
+                env.CreateFile(redistListDirectory, "FrameworkList.xml", redistListContents);
+
+                string targetFrameworkMoniker = "MyFramework, Version=v4.1";
+                MockEngine engine = new MockEngine();
+                GetReferenceAssemblyPaths getReferencePaths = new GetReferenceAssemblyPaths();
+                getReferencePaths.BuildEngine = engine;
+                getReferencePaths.TargetFrameworkMoniker = targetFrameworkMoniker;
+                getReferencePaths.RootPath = env.CreateFolder().FolderPath;
+                getReferencePaths.RootPath = frameworkRootDir;
+                getReferencePaths.TargetFrameworkFallbackSearchPaths = $"/foo/bar;{frameworkRootDir}";
+                getReferencePaths.Execute();
+                string[] returnedPaths = getReferencePaths.ReferenceAssemblyPaths;
+                string displayName = getReferencePaths.TargetFrameworkMonikerDisplayName;
+                Assert.Equal(1, returnedPaths.Length);
+                Assert.True(returnedPaths[0].Equals(framework41Directory.FolderPath, StringComparison.OrdinalIgnoreCase));
+                Assert.Equal(0, engine.Log.Length); // "Expected the log to contain nothing"
+                Assert.True(displayName.Equals(".NET Framework 4.1", StringComparison.OrdinalIgnoreCase));
+            }
+        }
+
     }
 }
 #endif

src/Tasks/GetReferenceAssemblyPaths.cs

@@ -54,6 +54,18 @@ public class GetReferenceAssemblyPaths : TaskExtension
         /// </summary>
         private string _rootPath;
 
+        /// <summary>
+        /// Target frameworks are looked up in @RootPath. If it cannot be found
+        /// there, then paths in @TargetFrameworkFallbackSearchPaths
+        /// are used for the lookup, in order. This can have multiple paths, separated
+        /// by ';'
+        /// </summary>
+        public string TargetFrameworkFallbackSearchPaths
+        {
+            get;
+            set;
+        }
+
         /// <summary>
         /// By default GetReferenceAssemblyPaths performs simple checks
         /// to ensure that certain runtime frameworks are installed depending on the
@@ -237,7 +249,7 @@ public override bool Execute()
 
             try
             {
-                _tfmPaths = GetPaths(_rootPath, moniker);
+                _tfmPaths = GetPaths(_rootPath, TargetFrameworkFallbackSearchPaths, moniker);
 
                 if (_tfmPaths != null && _tfmPaths.Count > 0)
                 {
@@ -248,7 +260,7 @@ public override bool Execute()
                 // There is no point in generating the full framework paths if profile path could not be found.
                 if (targetingProfile && _tfmPaths != null)
                 {
-                    _tfmPathsNoProfile = GetPaths(_rootPath, monikerWithNoProfile);
+                    _tfmPathsNoProfile = GetPaths(_rootPath, TargetFrameworkFallbackSearchPaths, monikerWithNoProfile);
                 }
 
                 // The path with out the profile is just the reference assembly paths.
@@ -278,18 +290,15 @@ public override bool Execute()
         /// <summary>
         /// Generate the set of chained reference assembly paths
         /// </summary>
-        private IList<String> GetPaths(string rootPath, FrameworkNameVersioning frameworkmoniker)
+        /// FIXME: do we really need the new arg? or should we just use the property?
+        private IList<String> GetPaths(string rootPath, string targetFrameworkFallbackSearchPaths, FrameworkNameVersioning frameworkmoniker)
         {
-            IList<String> pathsToReturn = null;
-
-            if (String.IsNullOrEmpty(rootPath))
-            {
-                pathsToReturn = ToolLocationHelper.GetPathToReferenceAssemblies(frameworkmoniker);
-            }
-            else
-            {
-                pathsToReturn = ToolLocationHelper.GetPathToReferenceAssemblies(rootPath, frameworkmoniker);
-            }
+            IList<String> pathsToReturn = ToolLocationHelper.GetPathToReferenceAssemblies(
+                                                frameworkmoniker.Identifier,
+                                                frameworkmoniker.Version.ToString(),
+                                                frameworkmoniker.Profile,
+                                                rootPath,
+                                                targetFrameworkFallbackSearchPaths);
 
             if (!SuppressNotFoundError)
             {

src/Tasks/Microsoft.Common.CurrentVersion.targets

@@ -80,7 +80,7 @@ Copyright (C) Microsoft Corporation. All rights reserved.
     <!-- The FrameworkPathOverride is required for the inproc visual basic compiler to initialize when targeting target frameworks less than 4.0. If .net 2.0 is not installed then the property value above will not provide the location
              of mscorlib. This is also true if the build author overrides this property to some other directory which does not contain mscorlib.dll. In the case we cannot find mscorlib.dll at the correct location
              we need to find a directory which does contain mscorlib to allow the inproc compiler to initialize and give us the chance to show certain dialogs in the IDE (which only happen after initialization).-->
-    <FrameworkPathOverride Condition="'$(EnableFrameworkPathOverride)' != 'false' And '$(FrameworkPathOverride)' == ''">$([Microsoft.Build.Utilities.ToolLocationHelper]::GetPathToStandardLibraries($(TargetFrameworkIdentifier), $(TargetFrameworkVersion), $(TargetFrameworkProfile), $(PlatformTarget), $(TargetFrameworkRootPath)))</FrameworkPathOverride>
+    <FrameworkPathOverride Condition="'$(EnableFrameworkPathOverride)' != 'false' And '$(FrameworkPathOverride)' == ''">$([Microsoft.Build.Utilities.ToolLocationHelper]::GetPathToStandardLibraries($(TargetFrameworkIdentifier), $(TargetFrameworkVersion), $(TargetFrameworkProfile), $(PlatformTarget), $(TargetFrameworkRootPath), $(TargetFrameworkFallbackSearchPaths)))</FrameworkPathOverride>
     <FrameworkPathOverride Condition="'$(EnableFrameworkPathOverride)' != 'false' And !Exists('$(FrameworkPathOverride)\mscorlib.dll')">$(MSBuildFrameworkToolsPath)</FrameworkPathOverride>
   </PropertyGroup>
 
@@ -1196,6 +1196,7 @@ Copyright (C) Microsoft Corporation. All rights reserved.
         Condition="'$(TargetFrameworkMoniker)' != '' and ('$(_TargetFrameworkDirectories)' == '' or '$(_FullFrameworkReferenceAssemblyPaths)' == '')"
         TargetFrameworkMoniker="$(TargetFrameworkMoniker)"
         RootPath="$(TargetFrameworkRootPath)"
+        TargetFrameworkFallbackSearchPaths="$(TargetFrameworkFallbackSearchPaths)"
         BypassFrameworkInstallChecks="$(BypassFrameworkInstallChecks)"
         >
       <Output TaskParameter="ReferenceAssemblyPaths" PropertyName="_TargetFrameworkDirectories"/>