Instrumented tests crash with AndroidJUnitRunner not being found on classpath on Android 8.0 emulator

[matejdro]

Steps to reproduce the bug:

1. Open TestProject.zip
2. Run included ExampleInstrumentedTest test
3. Tests will crash with following exception (in Logcat)

E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.matejdro.testproject, PID: 15073
    java.lang.RuntimeException: Unable to instantiate instrumentation ComponentInfo{com.matejdro.testproject.test/android.support.test.runner.AndroidJUnitRunner}: java.lang.ClassNotFoundException: Didn't find class "android.support.test.runner.AndroidJUnitRunner" on path: DexPathList[[],nativeLibraryDirectories=[/system/lib, /vendor/lib]]
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5730)
        at android.app.ActivityThread.-wrap1(Unknown Source:0)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1661)
        at android.os.Handler.dispatchMessage(Handler.java:105)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6541)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)
     Caused by: java.lang.ClassNotFoundException: Didn't find class "android.support.test.runner.AndroidJUnitRunner" on path: DexPathList[[],nativeLibraryDirectories=[/system/lib, /vendor/lib]]
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:93)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5728)
        at android.app.ActivityThread.-wrap1(Unknown Source:0) 
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1661) 
        at android.os.Handler.dispatchMessage(Handler.java:105) 
        at android.os.Looper.loop(Looper.java:164) 
        at android.app.ActivityThread.main(ActivityThread.java:6541) 
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767) 

When removing testInstrumentationRunnerArgument line and everything JUnit5-related from dependencies, tests start working again

What I did to create this test project:

1. Create new Android project with Kotlin support and no activity
2. Bump gradle version to 4.7 to allow plugin to work
3. Copy paste all instrumented test gradle stuff from README to the gradle file

So it is basically bare minimum that is supposed to work. Am I doing something wrong here or is this a bug?

If it helps, I reproduced this on Nexus 5X x86 emulator with Android 8.0 (Google Play) System image

[mannodermaus]

I cannot seem to reproduce this behavior on my end. The setup in your attached project runs fine for me, and its configuration looks OK as well:

Testing started at 17:54 ...

11/21 17:54:29: Launching All Tests
$ adb push /path/to/TestProject/app/build/outputs/apk/debug/app-debug.apk /data/local/tmp/com.matejdro.testproject
$ adb shell pm install -t -r "/data/local/tmp/com.matejdro.testproject"
Success
APK installed in 2 s 141 ms
$ adb push /path/to/TestProject/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk /data/local/tmp/com.matejdro.testproject.test
$ adb shell pm install -t -r "/data/local/tmp/com.matejdro.testproject.test"
Success
APK installed in 2 s 373 ms
Running tests

$ adb shell am instrument -w -r   -e debug false -e runnerBuilder de.mannodermaus.junit5.AndroidJUnit5Builder com.matejdro.testproject.test/android.support.test.runner.AndroidJUnitRunner
Client not ready yet..
Started running tests
Tests ran to completion.

The requirement on the AndroidJUnitRunner is enforced by the Android environment, and the JUnit 5 support simply hooks into that through a test instrumentation runner argument.

One thing that came to my mind: Do you have android.useAndroidX=true and/or the Jetifier enabled on a global level or something? That might cause a problem with the old package name. Then again, if everything works for you after removing JUnit 5, it's unlikely...

Furthermore, some more general things to throw at the project. Could you try:

1. ./gradlew clean connectedCheck?
2. File > Invalidate Caches / Restart...?

[matejdro]

I've done some more testing and it appears that the issue is contained to the Oreo emulator.

I could not reproduce it on Pie emulator, but could do it on any emulator running API 26 Google Play Intel x86 Atom System Image (revision 7).

[mannodermaus]

Hmm, thanks for getting back with more info. I'm going to try the same shortly. Still thinking about the potential clash regarding androidx and android.support. Which of the two are you using?

[matejdro]

there is only android.support, no androidx in the project from the first post

[mannodermaus]

I found something in the configuration of your test project that doesn't meet the requirements. The compilation target isn't set to Java 8 explicitly. I was able to reproduce the problem on the AVD you mentioned, and after I set the project to Java 8 & clean build, it works:

android {
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
}

Could you try this on your end?

[matejdro]

Thanks for checking this out.

Is this requirement mentioned somewhere in the README.MD? I could not find it.

Anyway, with your snipped added, tests works. However, this option is a bit problematic, because it force-enables desugaring with the Android plugin for all code (main and test). Only tests contain Java 8 code (in JUnit5 dependencies), main code is Kotlin, so running desugaring on main code will just waste compile time.

I wonder if there is a way to enable this just for tests? I tried following without success:

afterEvaluate {
    compileDebugAndroidTestJavaWithJavac {
        targetCompatibility = 1.8
        sourceCompatibility = 1.8
    }
}

[mannodermaus]

Hmm, not entirely sure how Gradle works here, but I'm afraid that source & target compatibility are a project-level setting, unfortunately. I guess you could make a separate module only for the tests, and specify it there?

The wiki mentions that Java 8 is required, however I'll make sure to append a distinct section on the setup page for instrumentation tests, too. Thanks for raising awareness about this!

[Talal916]

Hi there, did you figure out a confirmed solution? I'm running into the exact same problem with a different project of my own.

[mannodermaus]

@Talal916, do you have a link to the project in question you can share? In general, make sure to enable Java 8 source & target compatibility on your project, connect the instrumentation runner (Guide) and you should be good to go.

[Talal916]

Hi, I was trying to test https://github.com/romannurik/muzei this Android application. I ended up getting it to work but not sure what I did got it working. Thank you though for the guide, I appreciate it. Will look into it for future reference.