Skip to content

WinAppDriver in CI with Azure Pipelines

Hassan Uraizee edited this page May 2, 2019 · 1 revision

Prerequisites to run WinAppDriver in CI

The following are prequisites for running UI tests with WinAppDriver in Azure DevOps:

  1. An agent running on Windows 10 configured as interactive is required.

    • The following hosted agents are supported: HostedVS2019 and HostedVS2017.
    • If the hosted agents do not meet your requirement, try using a private agent. More information on this below.
  2. Use Azure Pipelines - get started today for free.

WinAppDriver Task for Azure Pipelines

There's now a dedicated WinAppDriver Pipelines Task available on the Azure Marketplace to help you easily enable and configure WinAppDriver from inside your DevOps Pipeline. To get started, install the WinAppDriver task onto your DevOps organization from here, or you can search through the Add Tasks menu from inside your Pipeline Editor -

Once installed, the WinAppDriver task can be added to your pipeline.

It is recommended to place the WinAppDriver task in conjunction with a separate Test Runner utility - in this case our pipeline has been pre-equipped with the Visual Studio Test Runner Task to drive the test cases.

We can now configure the task - you can see that automatically the Task is named to "Start - WinAppDriver". It is not required to pass in any arguments, but it is recommended to set the System Resolution on Agent to 1080P - this will be especially important to declare on a hosted agent.

Once configured, it's recommended to add another instance of the WinAppDriver task, this time having it be configured to "Stop" WinAppDriver on the agent. The "Stop- WinAppDriver" instance of the task should be placed after the VS Test task, so that WinAppDriver is terminated on the agent once all the test cases have been executed.

Congratulations! Assuming all went well, you can now view the captured WinAppDriver logs from the Summary Panel of the "Close - WinAppDriver" instance of the task.

And the WinAppDriver logs,

Setting up your Build Pipeline

The scope of this section is currently limited to only setting up the WinAppDriver UI test cases inside your test pipeline. For additional information on deploying builds on VSTS, please refer to this article.

Inside our build definition, we create a new phase to handle our WinAppDriver UI Automation. This test phase will follow the tasks as outlined in the following build phase snippet -

  1. Since we’re using a repository with only UI Tests, we decided to create our build pipeline from start using our UWPControls test sample.

  2. The .NET Desktop schema serves as a sufficient template for getting started with deploying WAD through a build pipeline.

  3. Once created, your UIA Automation Phase should look like the following screenshot.

  1. Ensure that your interactive Azure VM agent is selected on your Agent Queue as seen below.

  2. Declare Test Assemblies – VsTest task, ensure you have the proper test assemblies declared. Since our build is producing our VS Test cases in the UWPControls.dll, we point towards that.

  3. Add the WinAppDriver Task to your Pipeline (see above)

  4. Ensure correct Agent is declared in the Agent Queue, and queue your pipeline!

Once queued, your agent should initiate build process and execute the WinAppDriver unit tests.

Getting Started with a Sample Build Pipeline

WinAppDriver integration on Visual Studio Team Services (VSTS) requires deploying a custom agent on a Windows 10 based machine. This can be an Azure VM or your own personal computer. Setting up the agent on your target-machine will allow that machine to be used by VSTS to build, deploy, and run your solutions with WinAppDriver UI automation.

Before setting up the agent on your Azure VM, ensure the following prerequisites are met:

  • Machine is running Windows 10 (Home, Professional, or Server 2016).
  • WinAppDriver (V1.0+) has been installed.
  • Visual Studio 2017 installed with VS Unit Tests added.

Once met, complete the following steps to configure the agent:

  1. Preparing Permissions

    • You will first have to authenticate the agent with your VSTS personal access token (PAT). Detailed instructions can be found here.
      • PAT can be created from security panel, found on the following address:
      • Make sure to save the PAT somewhere secure and accessible.
  2. Downloading agent

    • On your target machine, download the agent as outlined below. Detailed instructions found here.
      • For VSTS, use the following address (this will require advanced user privileges):
      • Press the Download agent button and have Windows selected as operating system.
      • Once downloaded, you can now begin to configure the agent.
  3. Configuring the Agent

    • Run config.cmd to initiate configuration on your target-machine.
    • Enter “N” when asked to run agent as service -- this will set the agent to interactive mode.
    • It is recommended to enable auto-startup when prompted by the config tool.

For further instructions in deploying a VSTS agent, please refer to the official documentation here.

Once the agent is fully configured and ready to go, it can be used for the Build or VSTS release pipleines integrating with UI Automation testing.

You can’t perform that action at this time.