WinUI - Test Infra

[azchohfi]

WCT is now sharing WinUI's test infrastructure. This is the continuation of the work on this PR microsoft/microsoft-ui-xaml#3212.

PR Type

What kind of change does this PR introduce?

• Feature
• Build or CI related changes

What is the current behavior?

No UI Tests.

What is the new behavior?

Added simple UI Tests for the WCT that use the same shared infrastructure used by WinUI. This will enable us to move controls/helpers in an easier way from the WCT->WinUI in the future.

PR Checklist

Please check if your PR fulfills the following requirements:

• Tested code with current supported SDKs
• Pull Request has been submitted to the documentation repository instructions. Link:
• Sample in sample app has been added / updated (for bug fixes / features)
  • Icon has been created (if new sample) following the Thumbnail Style Guide and templates
• Tests for the changes have been added (for bug fixes / features) (if applicable)
• Header has been added to all new source files (run build/UpdateHeaders.bat)
• Contains NO breaking changes

[ghost]

Thanks azchohfi for opening a Pull Request! The reviewers will test the PR and highlight if there is any conflict or changes required. If the PR is approved we will proceed to merge the pull request 🙌

[michael-hawker]

@azchohfi

Looks like it can't find a package for the test run?

Summary of Errors Outside of Tests:
    Error: Installation failed!
    Error: Package installation ActivityId = 37bbe073-bde9-0001-1ddb-c117428bd601
    Error: Package installation ErrorText = Windows cannot install package 3568ebdf-5b6b-4ddd-bb17-462d614ba50f_1.0.0.0_x86__gspb8g6x97k2t because this package depends on a framework that could not be found. Provide the framework "Microsoft.UI.Xaml.2.4" published by "CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US", with neutral or x86 processor architecture and minimum version 2.42005.18002.0, along with this package to install. The frameworks with name "Microsoft.UI.Xaml.2.4" currently installed are: {}
    Error: Package installation ExtendedErrorCode = System.Exception: Package failed updates, dependency or conflict validation. (0x80073CF3)
    Error: System.Exception: Failed to install Test Appx Package: Windows cannot install package 3568ebdf-5b6b-4ddd-bb17-462d614ba50f_1.0.0.0_x86__gspb8g6x97k2t because this package depends on a framework that could not be found. Provide the framework "Microsoft.UI.Xaml.2.4" published by "CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US", with neutral or x86 processor architecture and minimum version 2.42005.18002.0, along with this package to install. The frameworks with name "Microsoft.UI.Xaml.2.4" currently installed are: {}
   at Windows.UI.Xaml.Tests.MUXControls.InteractionTests.Infra.TestAppInstallHelper.InstallTestAppIfNeeded(String deploymentDir, String packageName, String packageFamilyName, String appInstallerName)
   at Windows.UI.Xaml.Tests.MUXControls.InteractionTests.Infra.Application.Launch(String deploymentDir)
   at Windows.UI.Xaml.Tests.MUXControls.InteractionTests.Infra.Application.Initialize(Boolean doLaunch, String deploymentDir)
   at Windows.UI.Xaml.Tests.MUXControls.InteractionTests.Infra.TestEnvironment.Initialize(TestContext testContext, TestApplicationInfo testAppInfo)
   at UITests.Tests.Tests.ClassInitialize(TestContext testContext) in /_/UITests/UITests.Tests.Shared/Tests.cs:line 86
    Error: TAEF: Setup fixture 'ClassInitialize' for the scope 'UITests.Tests.Tests' failed.

[azchohfi]

Yeah, that started to happen when I added a reference to the Layout project... I'm fixing this on WinUI first.

[michael-hawker]

@azchohfi it prompted me to install the app when trying to run the MSTest project tests from VS, is this expected? (Assuming this doesn't happen during the CI?)

[azchohfi]

Yes, this is expected. The CI uses the TAEF tests, and installs the cert automatically. It works fine on the CI.

[michael-hawker]

@azchohfi is there a way we can have the local scripts for MS Test have less enter-to-confirm steps (there were another 3-4 after this first one here)? Is there a reason we can't deploy the app as normal through the VS process?

At least it's a one-time step thing, but we should probably document this in our Wiki on this subject too.

[azchohfi]

Its a one time only thing, and we have to do it. I don't know if there is a way around it.

[michael-hawker]

This is great @azchohfi thanks for getting this all setup!

Can we not see these with the rest of our Unit Tests in the Tests tab under Dev Ops?

I saw the new build step about it reporting, but figured they should be searchable here too, eh?

[michael-hawker]

Do we still need this?

[azchohfi]

We do still need this. Its where the package is published. It is a public feed.

[michael-hawker]

@azchohfi do we need to run the TestResultsPublisher.exe command to upload the .wtl file to the Azure dashboard? I noticed that seemed to be a difference between the Unit Test run and the UI Test run?

[azchohfi]

That wouldn't work. We would need to convert the wtl file to something that ADO understands, and that is not so simple. I know the Terminal team did exactly this, but they are running their tests on Helix, so the logs live there and their code was doing something that was highly coupled with Helix, so I couldn't easily port the code. I think this is a technical debt that we can handle in the future, since its not that simple and adds not much value right now.

[ghost]

Hello @michael-hawker!

Because this pull request has the auto merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

[michael-hawker]

As discussed this is good and we'll continue the discussion on improving the setup of individual tests in #3512