Adding initial Helix support

Documentation/developer-guide.md

@@ -23,6 +23,34 @@ We use the following workflow for building and testing features and fixes.
 
 You first need to [Fork](https://github.com/dotnet/corefx/wiki/Checking-out-the-code-repository#fork-the-repository) and [Clone](https://github.com/dotnet/corefx/wiki/Checking-out-the-code-repository#clone-the-repository) this WPF repository. This is a one-time task.
 
+
+### Running DRTs locally ###
+In order to run the set of DRTs on your local machine, pass the `-test` parameter to the `build.cmd` script. At the end of the run, you should see something like this:
+
+```
+  A total of 1 test Infos were processed, with the following results.
+   Passed: 1
+   Failed (need to analyze): 0
+   Failed (with BugIDs): 0
+   Ignore: 0
+
+```
+If there were any failures, you can cd into $(RepoRoot)\artifacts\test\$(Configuration)\$(Platform)\Test and run the tests manually with the `/debugtests` flag using the `RunDrts.cmd` script. Note that you do not run the `RunDrtsDebug` script, as this will debug the test infrastructure, `QualityVault`. When you pass the `/debugtests` flag, a cmd window will open where you can open the test executable in Visual Studio and debug it. When the cmd pops up, you will see instructions for debugging using a few different commands, however these commands will enable you to debug the `Simple Test Invocation` executable, `sti.exe`, which simply launches the test executable you are most likely interested in debugging. Using `DrtXaml.exe` as an example, this is how you can debug the test executable. Any MSBuild style properties should be replaced with actual values:
+
+1. `$(RepoRoot)\artifacts\test\$(Configuration)\$(Platform)\Test\RunDrts.cmd /name=DrtXaml /debugtests`
+2. Enter following command into the cmd window that pops up:
+`"%ProgramFiles%\Microsoft Visual Studio\2019\Preview\Common7\IDE\devenv.exe" DrtXaml.exe`
+3. Once Visual Studio is open, go to `Debug-> DrtXaml Properties` and do the following:
+    - Manually change the `Debugger Type` from `Auto` to `Mixed (CoreCLR)`.
+    - Change the `Environment` from `Default` to a custom one that properly defines the `DOTNET_ROOT` variable so that the host is able to locate the install of `Microsoft.NETCore.App`.
+      - x86 (Default): Name: `DOTNET_ROOT(x86)` Value: `$(RepoRoot).dotnet\x86`
+      - x64 (/p:Platform=x64): Name: `DOTNET_ROOT` Value: `$(RepoRoot).dotnet` 
+4. From there you can F5 and the test will execute.
+
+*Note: To run a specific test, you can pass the name of the test like this: `/name=DrtXaml`. The names of these tests are contained in DrtList.xml.*
+
+**NOTE: This requires being run from an admin window at the moment. Removing this restriction is tracked by https://github.com/dotnet/wpf/issues/816. **
+
 ### Testing Locally built WPF assemblies (excluding PresentationBuildTasks)
 This section of guide is intended to discuss the different approaches for ad-hoc testing of WPF assemblies,
 and not automated testing. There are a few different ways this can be done, and for the most part,
@@ -162,3 +190,4 @@ Follow the steps defined [here](https://github.com/dotnet/arcade/blob/master/Doc
 * [up-for-grabs WPF issues](https://github.com/dotnet/wpf/issues?q=is%3Aopen+is%3Aissue+label%3Aup-for-grabs)
 * [easy WPF issues](https://github.com/dotnet/wpf/issues?utf8=%E2%9C%93&q=is%3Aopen+is%3Aissue+label%3Aeasy)
 * [Code generation in dotnet/wpf](codegen.md)
+* [Testing in Helix](testing-in-helix.md)

Documentation/testing-in-helix.md

@@ -0,0 +1,12 @@
+# Testing in Helix
+
+I'd recommend seeing the official Helix [readme](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Helix/Sdk/Readme.md) if you are interested in some of the general Helix concepts. I'll briefly outline what we are doing that is a bit unique:
+
+1. Helix has built-in support for running xUnit tests. Since we are not using xUnit, we have to manually setup our machines so that they work with QualityVault and STI. During the build, we create a payload directory that contains the infrastructure we need. A single project (in our case `DrtXaml`) is responsible for creating this directory (see instances of the MSBuild property `CreateTestPayload`).
+2. After the build is done, we utilize Arcade's `AfterSolutionBuild.targets` extension point to finish creating the rest of the payload if the `-test` parameter is passed to the build. Here we add the just built DRTs and if `-ci` was **not** passed into to the build, run the tests. 
+3. Helix allows you to specify a `HelixCorrelationPayload` directory, where this directory gets deployed to the Helix machine, and is made available in your various helix commands with the `HELIX_CORRELATION_PAYLOAD` environment variable. We use the payload directory created described above.
+4. Helix and Azure Pipelines can report xUnit logs, so we will be updating QualityVault to produce an xUnit compatible log. We will then need to copy that log to a known location for it to be picked up. This location can be in any subfolder of the `HelixWorkItem` working directory. 
+
+## How we are running tests
+1. Currently, the `HelixQueues` that we selected are Windows.10.Amd64.Open;Windows.7.Amd64.Open;Windows.10.Amd64.ServerRS5.Open. Essentially, this translates to: "Latest Windows 10", "Windows 7", and "Windows 10 RS5 Server" addition. This enables tests to run on some of the most interesting and important SKUs, without overloading the Helix servers and/or making CI runs take an unnecessarily long time to run.
+2. In a similar fashion, we only run test passes for Release builds, mainly to save time and resources running duplicate tests in a near-identical environment.

Microsoft.Dotnet.Wpf.sln

@@ -81,9 +81,9 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "System.Printing-ref", "src\
 EndProject
 Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Tests", "Tests", "{B0EFDB12-C931-4E7F-A6C2-D4AC111D7EDF}"
 EndProject
-Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DrtXaml.Tests", "src\Microsoft.DotNet.Wpf\test\DRT\DrtXaml\DrtXaml\DrtXaml.Tests.csproj", "{FAB114A2-2C6C-4372-A6BB-BC087B646E3A}"
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DrtXaml", "src\Microsoft.DotNet.Wpf\test\DRT\DrtXaml\DrtXaml\DrtXaml.csproj", "{FAB114A2-2C6C-4372-A6BB-BC087B646E3A}"
 EndProject
-Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "TestServices", "src\Microsoft.DotNet.Wpf\test\Common\TestServices\TestServices.csproj", "{387F3700-8C0B-4CEC-A68A-1725F656A249}"
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "TestServices", "src\Microsoft.DotNet.Wpf\test\Common\DRT\TestServices\TestServices.csproj", "{387F3700-8C0B-4CEC-A68A-1725F656A249}"
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "XamlTestClasses", "src\Microsoft.DotNet.Wpf\test\DRT\DrtXaml\XamlTestClasses\XamlTestClasses.csproj", "{3C1FC36C-E3E6-4EED-9ECA-CFF2EB950486}"
 EndProject

eng/AfterSolutionBuild.targets

@@ -0,0 +1,3 @@
+<Project>
+    <Import Project="$(MSBuildThisFileDirectory)WpfArcadeSdk\tools\RunDrtsLocal.targets" Condition="'$(Test)'=='True'" />
+</Project>

eng/Version.Details.xml

@@ -1,99 +1,103 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Dependencies>
   <ProductDependencies>
-    <Dependency Name="Microsoft.Private.Winforms" Version="4.8.0-preview7.19304.2">
+    <Dependency Name="Microsoft.Private.Winforms" Version="4.8.0-preview7.19305.1">
       <Uri>https://github.com/dotnet/winforms</Uri>
-      <Sha>f59e9a9dde144098bb58365f210dfdf4acc9e693</Sha>
+      <Sha>e65ecacdf2da06f11c847571b467f632393bd060</Sha>
     </Dependency>
   </ProductDependencies>
   <ToolsetDependencies>
-    <Dependency Name="Microsoft.Win32.Registry" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="Microsoft.Win32.Registry" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.CodeDom" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.CodeDom" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Configuration.ConfigurationManager" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Configuration.ConfigurationManager" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Diagnostics.EventLog" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Diagnostics.EventLog" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.DirectoryServices" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.DirectoryServices" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Drawing.Common" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Drawing.Common" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Reflection.Emit" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Reflection.Emit" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Reflection.MetadataLoadContext" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Reflection.MetadataLoadContext" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Security.AccessControl" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Security.AccessControl" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Security.Cryptography.Xml" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Security.Cryptography.Xml" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Security.Permissions" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Security.Permissions" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Security.Principal.Windows" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Security.Principal.Windows" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="System.Windows.Extensions" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.Windows.Extensions" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.NETCore.App" Version="3.0.0-preview7-27804-03">
+    <Dependency Name="Microsoft.NETCore.App" Version="3.0.0-preview7-27805-02">
       <Uri>https://github.com/dotnet/core-setup</Uri>
-      <Sha>a620d0dba57cd18e49fa686e28e18369be23d067</Sha>
+      <Sha>2e54d1f6a6a9126126d6c9f8d730489892acf730</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19304.23">
+    <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19305.13">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>c9ab9c47d6c66fb9ad89e331900e06bed7463904</Sha>
+      <Sha>d05c046913964e880be08b804e7249f297617c2f</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.DotNet.CodeAnalysis" Version="1.0.0-beta.19304.23">
+    <Dependency Name="Microsoft.DotNet.CodeAnalysis" Version="1.0.0-beta.19305.13">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>c9ab9c47d6c66fb9ad89e331900e06bed7463904</Sha>
+      <Sha>d05c046913964e880be08b804e7249f297617c2f</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.NETCore.Platforms" Version="3.0.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="Microsoft.NETCore.Platforms" Version="3.0.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.DotNet.Wpf.DncEng" Version="4.8.0-preview7.19304.6">
+    <Dependency Name="Microsoft.DotNet.Wpf.DncEng" Version="4.8.0-preview7.19305.10">
       <Uri>https://dev.azure.com/dnceng/internal/_git/dotnet-wpf-int</Uri>
-      <Sha>24fbf672601b7b9c088abed2e850bfe14a69875f</Sha>
+      <Sha>76e5819b43718069ef5710a08032b616de5d754c</Sha>
     </Dependency>
-    <Dependency Name="System.IO.Packaging" Version="4.6.0-preview7.19304.7" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="System.IO.Packaging" Version="4.6.0-preview7.19305.9" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/corefx</Uri>
-      <Sha>2469a91c2aa68f9cb6b0f18f75dffc673a1f7895</Sha>
+      <Sha>5c2ba765f55462dcc5e00179a34398fc70bf37b7</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.NETCore.Runtime.CoreCLR" Version="3.0.0-preview7.19304.2" CoherentParentDependency="Microsoft.NETCore.App">
+    <Dependency Name="Microsoft.NETCore.Runtime.CoreCLR" Version="3.0.0-preview7.19305.2" CoherentParentDependency="Microsoft.NETCore.App">
       <Uri>https://github.com/dotnet/coreclr</Uri>
-      <Sha>89df4b9d928c7f21550d487328f5db000a498bdf</Sha>
+      <Sha>10df20ed3ff0208b3f16f79d5062662a8827f579</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.NETCore.ILDAsm" Version="3.0.0-preview7.19304.2" CoherentParentDependency="Microsoft.NETCore.Runtime.CoreCLR">
+    <Dependency Name="Microsoft.NETCore.ILDAsm" Version="3.0.0-preview7.19305.2" CoherentParentDependency="Microsoft.NETCore.Runtime.CoreCLR">
       <Uri>https://github.com/dotnet/coreclr</Uri>
-      <Sha>89df4b9d928c7f21550d487328f5db000a498bdf</Sha>
+      <Sha>10df20ed3ff0208b3f16f79d5062662a8827f579</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.NETCore.ILAsm" Version="3.0.0-preview7.19304.2" CoherentParentDependency="Microsoft.NETCore.Runtime.CoreCLR">
+    <Dependency Name="Microsoft.NETCore.ILAsm" Version="3.0.0-preview7.19305.2" CoherentParentDependency="Microsoft.NETCore.Runtime.CoreCLR">
       <Uri>https://github.com/dotnet/coreclr</Uri>
-      <Sha>89df4b9d928c7f21550d487328f5db000a498bdf</Sha>
+      <Sha>10df20ed3ff0208b3f16f79d5062662a8827f579</Sha>
+    </Dependency>
+    <Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="2.0.0-beta.19278.1">
+      <Uri>https://github.com/dotnet/arcade</Uri>
+      <Sha>11f90a2a260422201895de58e57170131ab4efe7</Sha>
     </Dependency>
   </ToolsetDependencies>
 </Dependencies>