Skip to content

Background

Jump to bottom Edit New page
Jeremy Kao edited this page Aug 15, 2015 · 3 revisions

As a test developer, and because of multi-platform release strategy, chances are that you have to implement automated tests against Windows applications, in addition to Android, iOS, web and even OS X applications.

Here, Windows applications generally refers to applications based on various UI frameworks, including Win32, DirectUI, WinForms (Windows Forms), Silverlight, WPF (Windows Presentation Foundation) and XAML (Extensible Application Markup Language), etc. XAML can also be used in Windows Store apps and Windows Phone apps.

In addition, we should also take hybrid applications into account. They are applications having a native wrapper around a webview. For desktop applications, such as Slack and Spotify for Desktop, it is usually implemented with CEF (Chromium Embedded Framework).

Looking for available testing tools, you may find most of them, such as AutoIt, only support Win32 applications. For WinForms, WPF, Silverlight and XAML, the answer would be UI Automation (UIA).

There are testing tools based on UIA. White is an open-source framework, and Coded UI Tests (CUIT) is only available in Visual Studio Premium/Ultimate.

Take Twitter app as an example, here is a snippet (written in C#) showing the usage of UIA Client API:

var accField = app.FindFirst(TreeScope.Descendants,
    new PropertyCondition(AutomationElement.AutomationIdProperty, "UserNameTextBox"));
    ((ValuePattern)accField.GetCurrentPattern(ValuePattern.Pattern)).SetValue("username");

var pwdField = app.FindFirst(TreeScope.Descendants,
    new PropertyCondition(AutomationElement.AutomationIdProperty, "PasswordTextBox"));
    ((ValuePattern)pwdField.GetCurrentPattern(ValuePattern.Pattern)).SetValue("secret");

var loginBtn = app.FindFirst(TreeScope.Descendants,
    new PropertyCondition(AutomationElement.AutomationIdProperty, "SignInButton"));
    ((InvokePattern)loginBtn.GetCurrentPattern(InvokePattern.Pattern)).Invoke();

As you can see, you have to locate UI elements with a certain condition first. Before you can interact with one of them, you have to retrieve a specific pattern supported by the element.

The problem with the API is that it is relatively verbose and the API alone is not able to inject native input events. For example, ValuePattern.SetValue() does not trigger keyboard events, and InvokePattern.Invoke() does not move the mouse pointer and trigger a mouse click event. To simulate user interactions, we have to trigger native input events programmatically.

To make UIA work properly with the application under test (AUT), some security restrictions need to be taken into account as well. Because of the integrity levels (IL) introduced in Windows Vista, and the AppContainer sandbox introduced in Windows 8 and primarily used for store apps, UIA clients must start with UIAccess privilege to bypass User Interface Privilege Isolation (UIPI) to gain access to protected UI or communicate with higher IL processes.

To be granted the UIAccess privilege, an UIA client needs to:

  • Be built with a manifest file that includes the UIAccess flag.
  • Be signed with a certificate trusted by the system.
  • Be installed in a secure location that requires a UAC prompt to write to.

Refer to the following documents for more details:

Here is the architecture of WinAppDriver.

Architecture

Inspired by the design of Appium, what we want to do is encapsulate verbosity, security considerations and tricks, and expose a well-known WebDriver API to test developers. Because there is no existing open-source tools like this, we started the development of WinAppDriver in Dec, 2014.

In the beginning, it supports only store apps, and then the support for traditional desktop application is added later.

As the name suggests, it aims for being a single tool for test automation of all kinds of Windows applications, therefore the support for CEF-based desktop applications and Windows Phone apps are planned as well.

Here is the design philosophy of WinAppDriver:

  • As a testing tool, it is important to have as few dependencies as possible.
  • Application under test (AUT) and test code do not have to run on the same machine.
  • Test developers have freedom of choice in their programming languages and testing frameworks.
  • Inject native input events to simulate user interactions whenever it is possible.
  • Follow de facto standards and be compatible with Selenium and Appium client libraries (or language bindings).

With WinAppDriver, the previous snippet can be rewritten in Python and using WebDriver API.

from selenium.webdriver import Remote

desired_caps = {
    'platformName': 'WindowsModern',
    'packageName': '9E2F88E3.Twitter',
}

driver = Remote('http://your-winappdriver-server:4444/wd/hub', desired_caps)
# ... (omitted)
driver.find_element_by_id('UserNameTextBox').send_keys('username')
driver.find_element_by_id('PasswordTextBox').send_keys('secret')
driver.find_element_by_id('SignInButton').click()

Obviously, the test code becomes more succinct and easy to read. If you are already familiar with WebDriver API, apart from desired capabilities dedicated to WinAppDriver, it just looks like test code for web applications.

Here is the screen recording:

Twitter Sign-in

You may notice that UI elements that match a certain condition are temporarily highlighted. This is a feature of WinAppDriver to help visualize the search result. WinAppDriver also trys vary hard to simulate user interactions. For the text input, WinAppDriver sends a sequence of keyboard events, and therefore the cursor moves accordingly while typing. For the mouse click, WinAppDriver moves the pointer to that point and then sends a click event.

Most commands defined in JSON wire protocol that are relevant to desktop applications or store apps have been implemented. Hope that this tool will make our life more easier if we are asked to implement automated test against Windows applications.

Give it a try. To get started, follow the instructions described in Getting Started, install WinAppDriver and try it out.

Add a custom footer
Add a custom sidebar
Clone this wiki locally