UI Tests

What do we understand (and support) as UI test?

There are many ways testing an application from a user perspective. Manually testing means start the application and work with it. There may be checklist or scenarios to process, but this kind of tests is not failure safe, because the developer knows the weak points of an application and the workarounds for them. A better way is the automatically testing of the application.

In the Java universe exit a lot of test frameworks for Java based applications. Eclipse itself provides some test methods. Currently SWT is the most used graphical interface, the most popular related test framework is SWTbot. For this reasons BuildMonkey supports at this moment only SWTbot tests.

For more information about SWTbot please take a look to the official documentation at Eclipse SWTbot.

How do we support SWTbot tests?

BuildMonkey works from the Eclipse perspective, so UI tests are just additional OSGi bundles for the application and are processed by a special executor. If you want develop UI tests with SWTbot, do it from Eclipse and with the Eclipse tooling. UI tests bundles should not be part of the application but can be part of the target platform for easy overall access. If the tests running from Eclipse, then BuildMonkey can help you to make them executable in CI environments over Gradle.

BuildMonkey expects a runnable application. It may be useful separating some UI tests in different bundles and execute them in clean environments. For this reason the definition of the application must be done at the root project using the UITestDefinitionPlugin. The plugin will download or copy the application into the build folder of the root project and installs the target platform (with the UI test features and there dependencies). This features can be built with BuildMonkey tooling, especially P2DeployerPlugin. This prepared application is the base for each test run.

The execution of the tests happens on project level. This can be the ui test bundle, but must not be, because the tests are taken from the bundle defined in the UiTestPlugin configuration. The plugin will prepare the application inside the bundle and execute the gien test class. Unfortunally the eclipse runner for swt bot tests does not support multiple classes, so this class must be a test suite defining the test cases and test suites to execute. After the definition the tests can be run easily with

gradle uiTest

from the root project.

Example layout

The sample project has the following layout

|-feature-uitest
|    |
|    |-build
|    |    |
|    |    |-p2
|    |-de.buildmonkey.uitest
|    |-de.buildmonkey.testbase
|    |-build.gradle
|
|-product-myapp
|    |
|    |-de.buildmonkey.myapp.target
|    |    |
|    |    |-my.fancy.target
|    |-de.buildmonkey.myapp.product
|    |    |
|    |    |-build
|    |    |    |
|    |    |    |-MyApp-win32.zip
|    |    |    |-MyApp-linux.tar.gz
|    |    |    |-MyApp-macosx.tar.gz
|    |-de.buildmonkey.myapp.feature
|    |-build.gradle
|    |-settings.gradle

This means, the ui tests are a feature which will be deployed in a local p2 repository using P2DeployerPlugin with

gradlew p2Publish

This tests will be executed inside the product 'MyApp', which was built in the build folder of 'de.buildmonkey.myapp.product'.

Example Gradle scripts

With the layout explained before the two gradle files of the 'product-myapp' should be look like this.

settings.gradle:

rootProject.name = 'de.buildmonkey.myapp'
include('de.buildmonkey.myapp.feature')

This defines the root project name and myapp.feature as only sub project.

build.gradle:

buildscript {
	repositories {
		maven { url 'https://artifactory.monkey-works.de/artifactory/JCenter/' }
		maven { url 'https://artifactory.monkey-works.de/artifactory/MCenter/' }
		maven { url 'https://plugins.gradle.org/m2/' }
	}
	dependencies {
        classpath "de.monkeyworks.buildmonkey:gradle.dependency:0.4.11"
        classpath "de.monkeyworks.buildmonkey:gradle.pde:0.4.16"
	}
}

apply plugin: 'de.monkeyworks.buildmonkey.pde.TestDefinitionPlugin'
apply plugin: 'de.monkeyworks.buildmonkey.pde.UITestDefinitionPlugin'

uiTestBuild {
    applicationUrl = 'file:/pathToCommonFolder/productMyApp/de.buildmonkey.myapp.product/build'
    applicationName = 'MyApp_win32.zip'
    targetDefinition = file('./de.buildmonkey.myapp.target/my.fancy.target')
}

subprojects {
    apply plugin: 'de.monkeyworks.buildmonkey.pde.UiTestPlugin'

    uiTest {
		applicationName 'org.eclipse.swtbot.eclipse.junit.headless.swtbottestapplication'
		application 'de.buildmonkey.myapp.Application'
		testClass 'de.buildmonkey.uitest.UITests'
		testBundle 'de.buildmonkey.uitest'
		product 'MyApp.product'
		consoleLog true
		debug false
		debugPort 8998
		testTimeoutSeconds 3600
		isTestClass {
			return it.getName() != '_Template.class' && !(it.getName() =~ /.+Support.class/ && !it.getName().contains('$'))
		}
    }
}

With this setup the test suite 'de.buildmonkey.uitest.UITests' is executed as SWTbot test within the application MyApp using the dependencies defined in the target platform 'my.fancy.target'.