Instrumentation tests, Release builds and Proguard

[auras]

There's some behind the scenes thing going on with Instrumentation tests / androidTest and the source code which I don't believe is explained anywhere.

Test on debug builds run fine. On release builds there's something strange. Let's take this example:

I'm using gson in my instrumentation tests, directly imported with androidTestImplementation "com.google.code.gson:gson:${versions.gson}" or implicitly imported from the main apk, I have this error:

java.lang.NoSuchMethodError: No virtual method getAsJsonArray(Ljava/lang/String;)Lcom/google/gson/JsonArray; in class Lcom/google/gson/n; or its super classes (declaration of 'com.google.gson.n' appears in /data/app/com.mydomain.app-tr_IypDBUv-MWFcNwrKcyA==/base.apk!classes3.dex)

The classes, methods and fields exist in the androidTest.apk, and I even added -keep class com.google.gson.** { *; } to my test proguard file:

release {
            minifyEnabled true
            proguardFiles getDefaultProguardFile('proguard-android.txt'), "proguard-app.pro"
            testProguardFile "proguard-app-test.pro"

And yet, the code that runs from the androidTest.apk that uses the getAsJsonArray() method, doesn't look for the method in androidTest.apk but it looks for it in the release.apk.

How are we supposed to deal with this?

This happens with compileSdk/targetSdk 29 and minSdk 26 running on the android emulator api 29.

[brettchabot]

Its recommended not to use proguard with instrumentation tests. It will slow down the build significantly and be prone to errors like this.

Is gson a dependency of your app as well? If so, try adding the -keep class com.google.gson.** { *; } clause to the app proguard spec to see if that helps

[auras]

It does help for sure. I don't want to change the app provided though because it would miss the point of shrinking and I wouldn't be testing the actual apk that would be downloaded by users.

[brettchabot]

Could you apply the proguard in a debug flavor of the prod app instead, and have your instrumentation tests run against that?

I understand the desire to test what you ship, but IMHO instrumentation tests should effectively be grey box type tests.