Reliable Android Testing, at your service
Java Groovy
Latest commit 7fc208a Jan 21, 2017 @handstandsam handstandsam committed with drewhannay Readme Update: Calling out that manual APK installation is required. (#…
…49)

* Calling out that separate APK is required.

I was very confused on how this worked.  I didn't know how it was working, but assumed that the APK was auto-installed after I added the `androidTestCompile` gradle dependency and the necessary lines in my custom `AndroidJUnitRunner`.

The goal of this change is to point out that the separate APK is needed on the device to get Test Butler to Work.

* Updated wording for clarity.

README.md

Test Butler

Build Status

Reliable Android testing, at your service.

Motivation

Test Butler was inspired by the Google presentation "Going Green: Cleaning up the Toxic Mobile Environment". For more background, read the Test Butler announcement blog post.

What does it do?

  • Stabilizes the Android emulator, to help prevent test failures caused by emulator issues.
    • Disables animations so Espresso tests can run properly.
    • Disables crash & ANR dialogs so a system app misbehaving won't trigger a popup that causes your UI tests to fail.
    • Locks the keyguard, WiFi radio, and CPU to ensure they don't go to sleep unexpectedly while tests are running.
  • Handles changing global emulator settings and holds relevant permissions so your app doesn't have to.
    • Enable/disable WiFi: Test Butler allows tests to simulate a situation where WiFi is not connected or changes availability at some point.
    • Change device orientation: Tests can manually set the orientation of the device during test execution.
    • Set location services mode: Test Butler lets your code simulate different location services modes, like battery saver or high accuracy.
    • Set application locale: Tests can set a custom Locale object for their application to simulate running the app in another language.
    • Grant runtime permissions: Tests can grant Marshmallow's runtime permissions to their application directly from test code.

How does it work?

Test Butler is a two-part project. It includes an Android library that your test code can depend on, as well as a companion Android app apk that must be installed on your Android emulator before using Test Butler. You can build the Test Butler APK yourself from source, or download the binary from Bintray

The Test Butler library is a thin wrapper around an AIDL interface to give your tests a safe way to talk to the Test Butler app's service running in the background on the emulator.

The Test Butler app is signed using the system keystore for the Android emulator, so it is automatically granted signature-level permissions when installed. This means granting permissions via adb is not necessary. It also means that this app can only be installed on emulators that use the stock Android keystore!

Being a system app makes Test Butler much easier to use than existing Android solutions for changing emulator settings. To disable animations, you just need a single line of code in your app; no extra permissions in your debug manifest, no granting permissions via adb, no Gradle plugin to integrate.

Test Butler can even use permissions that can't be granted via adb, like the SET_ACTIVITY_WATCHER permission, which lets Test Butler disable crash & ANR dialogs during tests.

Any "gotchas" to look out for?

Only a couple (and they're minor). Test Butler adds a custom IActivityController to the system to be able to suppress crash & ANR dialogs. This technique is also used internally by the Monkey tool. Unfortunately, the implementation of the isUserAMonkey() method takes advantage of the fact that the Monkey class is the only thing inside Android that sets an IActivityController and returns true whenever one is set.

This means that isUserAMonkey() will return true while Test Butler is running! If your app uses this method to invoke different behavior during actual monkey testing, you may encounter issues while running tests with Test Butler. An easy fix is to create a helper method in your app to call instead of isUserAMonkey(), which returns false while instrumentation tests are running and calls through to the real isUserAMonkey() when the app is not being instrumented.

Also, you can use methods in the TestButler class in the @BeforeClass & @AfterClass of a test class. However, be warned that the test suite will not report that the Test Butler app is not installed, instead it will skip the test. This is due to a bug in Android testing support.

Download

Download the latest .apk and .aar via Maven:

    <dependency>
      <groupId>com.linkedin.testbutler</groupId>
      <artifactId>test-butler-library</artifactId>
      <version>1.2.0</version>
      <type>pom</type>
    </dependency>
    <dependency>
      <groupId>com.linkedin.testbutler</groupId>
      <artifactId>test-butler-app</artifactId>
      <version>1.2.0</version>
      <type>pom</type>
    </dependency>

or Gradle:

    androidTestCompile 'com.linkedin.testbutler:test-butler-library:1.2.0'

You can also download the apk file manually from Bintray if you prefer.

Getting Started

Install the Test Butler apk on your emulator prior to running tests, then add the following to your test runner class:

public class ExampleTestRunner extends AndroidJUnitRunner {
  @Override
  public void onStart() {
      TestButler.setup(InstrumentationRegistry.getTargetContext());
      super.onStart();
  }

  @Override
  public void finish(int resultCode, Bundle results) {
      TestButler.teardown(InstrumentationRegistry.getTargetContext());
      super.finish(resultCode, results);
  }
}

To change settings on the device from your tests, just use the methods in the TestButler class. For example:

@Before
public static void setupClass() {
  TestButler.setLocationMode(Settings.Secure.LOCATION_MODE_OFF);
}

@Test
public void verifyBehaviorWithNoLocation() {
  // ...
}

@After
public static void teardownClass() {
  TestButler.setLocationMode(Settings.Secure.LOCATION_MODE_HIGH_ACCURACY);
}