Android testing broken since Gradle 4.2 #3006

jaredsburrows · 2017-09-22T16:43:05Z

Based on this reddit post: https://www.reddit.com/r/androiddev/comments/71c1sz/gradle_42_released/ and https://issuetracker.google.com/issues/65869429.

Gradle 4.2 broke espresso/instrumentation tests for Android Gradle Plugin.

Expected Behavior

The tests should run.

Current Behavior

> Task :roomDatabase:connectedDebugAndroidTest
Starting 37 tests on 5x-26(AVD) - 8.0.0


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':roomDatabase:connectedDebugAndroidTest'.
> Could not initialize class com.android.build.gradle.internal.test.report.AllTestResults

Steps to Reproduce (for bugs)

./gradlew connectedDebugAndroidTest

Your Environment

Gradle 4.2-rc-2 (happens on 4.1 as well)
Gradle Android Plugin version 3.0.0-beta6 (happened on beta2 as well)
Compile SDK version: 26
Build tools version: 26.0.1
Sdk tools version: 26.1.0

Possible solution

Could we add a single unit test and instrumentation test case for Gradle? This way we can ensure the latest Gradle releases work with existing versions of the Android Gradle plugin.

The text was updated successfully, but these errors were encountered:

bmuschko · 2017-09-22T23:45:05Z

Might be a good enhancement to org.gradle.smoketests.AndroidPluginsSmokeTest. Would you be interested in contributing?

Is this happening with 4.2 GA as well? Would you consider this a regression in Gradle core? If yes, can you please provide a sample project?

jaredsburrows · 2017-09-22T23:45:58Z

Yes I would!

…

On Sep 22, 2017 4:45 PM, "Benjamin Muschko" ***@***.***> wrote: Might be a good enhancement to org.gradle.smoketests. AndroidPluginsSmokeTest. Would you be interested in contributing? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3006 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABqMSA1Fs9tBGuw41Oq_hfrGFaBKrUHMks5slEaJgaJpZM4Pg9VN> .

jaredsburrows · 2017-09-23T00:47:18Z

Currently 4.1 works well, I have an example test project here: https://github.com/jaredsburrows/android-gif-example.

I am not sure whether or not to consider this apart of core or not.

I can reproduce the error in this project/branch that is apart of that repo in a PR here: jaredsburrows/android-gif-search#41.

jaredsburrows · 2017-09-23T01:00:15Z

@bmuschko It looks like Xavier marked this issue as fixed in the Android tracker: https://issuetracker.google.com/issues/65869429. Maybe this was an AGP problem?

I still think that we should have some integration tests for this.

oehme · 2017-09-23T06:33:46Z

We definitely should update our tests to check against the latest agp versions.

SteffenL · 2017-09-23T11:32:23Z

Is Gradle 4.2 supposed to be compatible with AGP 2.3.x or only AGP 3.x?

I am asking because I got this issue with AGP 2.3.2 (also tried 2.3.3) after updating Gradle from 3.5 to 4.2.

jaredsburrows · 2017-09-28T01:58:05Z

@bmuschko @oehme Where is the .travis.yml file? How are you all updating the SDK for tests?

bmuschko · 2017-09-28T17:07:53Z

@jaredsburrows Thanks for providing the sample project. I could reproduce the issue you reported with your sample project and an emulator.

I believe to automate an integration test, we'd have a configured emulator first. When starting the test, we'd have to bring up the emulator, run the test and then tear it down again. I am currently looking into that.

Where is the .travis.yml file?

Which Travis file are you referring to? We are running the Android smoke tests on TeamCity.

How are you all updating the SDK for tests?

I believe that is done in the infrastructure of the agents but I'd have to check.

bmuschko · 2017-09-28T22:16:06Z

This is what I believe is happening:

When running the example project I get the following stack trace:

Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.android.build.gradle.internal.test.report.AllTestResults
        at com.android.build.gradle.internal.test.report.TestReport.loadModel(TestReport.java:60)
        at com.android.build.gradle.internal.test.report.TestReport.generateReport(TestReport.java:55)
        at com.android.build.gradle.internal.tasks.DeviceProviderInstrumentTestTask.runTests(DeviceProviderInstrumentTestTask.java:173)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.doExecute(DefaultTaskClassInfoStore.java:142)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:135)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:122)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:762)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:729)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:121)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:199)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:110)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:110)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:92)
        ... 29 more

The stack trace indicates that the class AllTestResults cannot be initialized. I couldn't find the Android source code online so I had to download the sources from the repository:

repositories {
    google()
    jcenter()
}

configurations {
    android
}

dependencies {
    android group: 'com.android.tools.build', name: 'gradle-core', version: '3.0.0-beta6', classifier: 'sources'
}

task copy(type: Copy) {
    from configurations.android
    into "$buildDir/libs"
}

The line 60 in TestReport creates a new instance of AllTestResults:

AllTestResults model = new AllTestResults();

We can find the following hierarchy of parent classes for AllTestResults:

AllTestResults extends CompositeTestResults extends TestResultModel.

The class TestResultModel instantiates a field of the Gradle core internal type DurationFormatter:

public static final DurationFormatter DURATION_FORMATTER = new DurationFormatter();

In Gradle 4.1 the class DurationFormatter lived in package org.gradle.reporting.
In Gradle 4.2 the class was moved to the package org.gradle.internal.logging.format and became an interface. The actual implementation is now TersePrettyDurationFormatter.

The change between 4.1 and 4.2 was a breaking change even though it was an internal API. Given that the class also became an interface I don't think we can just move back the class. I'd suggest that the Android plugin implements its own time formatting along the lines of org.gradle.internal.time.TimeFormatting instead of relying on an internal API that could easily be implemented in the plugin as well. The logic isn't really that complicated. In the long run we could think about exposing a public API for (test) report generation. That would also align we some of the goals we discussed before for plugin developers.

Apparently the Android team already fixed the issue though I do not know with what version and how. I left a comment in the bug report.

oehme · 2017-09-29T07:37:52Z

The Android team fixed it for 3.0, but 2.3 remains broken. I suggest we restore the old DurationFormatter class, add @Deprecated and remove it in Gradle 5.0. This would be a cheap fix and would allow Android 2.3 users (the vast majority) to enjoy Gradle 4.2.

oehme · 2017-09-29T12:23:43Z

@bmuschko I'll take this one over so you'll have more time for #2832 and #2837

bmuschko · 2017-09-29T12:40:39Z

@oehme I already started to refactor existing Android smoke tests and set up unit/instrumented tests in this branch bm/android-smoke-tests. You can continue to commit to that branch.

jaredsburrows · 2017-09-29T14:26:52Z

Is there a way to reproduce the issue without having to connect an Android device?

Sadly "android tests" and not "unit tests" have to run on a device or an emulator.

Which Travis file are you referring to?

Sorry, I just assumed Travis.

I believe that is done in the infrastructure of the agents but I'd have to check.

If I update the build tools/sdk, will it break your current build? I just want to make sure if we upgrade to the latest release, the build server is ready.

jaredsburrows · 2017-09-29T14:34:38Z

Awesome. Just now catching up on this thread, it looks like @bmuschko is going to go ahead and update the existing smoke tests?

oehme · 2017-09-29T14:39:56Z

Yes, the tests have been adjusted together with the fix, see the linked Pull Request.

bmuschko · 2017-09-29T14:41:38Z

@jaredsburrows I think there's more we can do to cover unit and instrumented testing. Feel free to have a look at my branch and create a PR to improve even more.

jaredsburrows · 2017-09-29T14:43:02Z

Thanks @oehme and @bmuschko.

If I were to add on to @bmuschko's branch, should I wait for both of your PRs to be merged in to master first?

bmuschko · 2017-09-29T14:45:07Z

@jaredsburrows I'd wait for @oehme's PR to be merged. After that you can create a new PR. I'd just copy/paste the changes you think are useful from my branch.

jaredsburrows · 2017-09-29T14:45:31Z

Sounds great, thanks!

oehme · 2017-09-29T15:43:23Z

The fix has been merged into release.

IgorGanapolsky · 2017-10-13T15:42:38Z

This issue seems to be resolved in Gradle 4.3-rc-1.

jaredsburrows · 2017-10-13T15:43:42Z

It was fixed in 4.2.1.

…

On Oct 13, 2017 8:43 AM, "Igor Ganapolsky" ***@***.***> wrote: This issue seems to be resolved in Gradle 4.3-rc-1. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3006 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABqMSE6VB2pMFj4o0TtuKcDa2XUXJMpfks5sr4UEgaJpZM4Pg9VN> .

bmuschko added a:chore Minor issue without significant impact from:contributor a:bug and removed a:chore Minor issue without significant impact labels Sep 22, 2017

jaredsburrows mentioned this issue Sep 23, 2017

Test if instrumentation tests still work in Gradle 4.2 jaredsburrows/android-gif-search#41

Closed

oehme added this to the 4.2.1 milestone Sep 28, 2017

oehme changed the title ~~[Test Request] - Android testing for unit/instrumentation tests~~ Android unit testing broken since Gradle 4.2 Sep 28, 2017

oehme added the affects-version:4.2 label Sep 28, 2017

oehme assigned bmuschko Sep 28, 2017

oehme assigned oehme and unassigned bmuschko Sep 29, 2017

oehme changed the title ~~Android unit testing broken since Gradle 4.2~~ Android testing broken since Gradle 4.2 Sep 29, 2017

oehme closed this as completed Sep 29, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Android testing broken since Gradle 4.2 #3006

Android testing broken since Gradle 4.2 #3006

jaredsburrows commented Sep 22, 2017

bmuschko commented Sep 22, 2017 •

edited

Loading

jaredsburrows commented Sep 22, 2017 via email

jaredsburrows commented Sep 23, 2017

jaredsburrows commented Sep 23, 2017

oehme commented Sep 23, 2017

SteffenL commented Sep 23, 2017

jaredsburrows commented Sep 28, 2017

bmuschko commented Sep 28, 2017 •

edited

Loading

bmuschko commented Sep 28, 2017 •

edited

Loading

oehme commented Sep 29, 2017 •

edited

Loading

oehme commented Sep 29, 2017

bmuschko commented Sep 29, 2017 •

edited

Loading

jaredsburrows commented Sep 29, 2017

jaredsburrows commented Sep 29, 2017

oehme commented Sep 29, 2017

bmuschko commented Sep 29, 2017

jaredsburrows commented Sep 29, 2017

bmuschko commented Sep 29, 2017

jaredsburrows commented Sep 29, 2017

oehme commented Sep 29, 2017

IgorGanapolsky commented Oct 13, 2017

jaredsburrows commented Oct 13, 2017 via email

Android testing broken since Gradle 4.2 #3006

Android testing broken since Gradle 4.2 #3006

Comments

jaredsburrows commented Sep 22, 2017

Expected Behavior

Current Behavior

Steps to Reproduce (for bugs)

Your Environment

Possible solution

bmuschko commented Sep 22, 2017 • edited Loading

jaredsburrows commented Sep 22, 2017 via email

jaredsburrows commented Sep 23, 2017

jaredsburrows commented Sep 23, 2017

oehme commented Sep 23, 2017

SteffenL commented Sep 23, 2017

jaredsburrows commented Sep 28, 2017

bmuschko commented Sep 28, 2017 • edited Loading

bmuschko commented Sep 28, 2017 • edited Loading

oehme commented Sep 29, 2017 • edited Loading

oehme commented Sep 29, 2017

bmuschko commented Sep 29, 2017 • edited Loading

jaredsburrows commented Sep 29, 2017

jaredsburrows commented Sep 29, 2017

oehme commented Sep 29, 2017

bmuschko commented Sep 29, 2017

jaredsburrows commented Sep 29, 2017

bmuschko commented Sep 29, 2017

jaredsburrows commented Sep 29, 2017

oehme commented Sep 29, 2017

IgorGanapolsky commented Oct 13, 2017

jaredsburrows commented Oct 13, 2017 via email

bmuschko commented Sep 22, 2017 •

edited

Loading

bmuschko commented Sep 28, 2017 •

edited

Loading

bmuschko commented Sep 28, 2017 •

edited

Loading

oehme commented Sep 29, 2017 •

edited

Loading

bmuschko commented Sep 29, 2017 •

edited

Loading