Limitation of ActivityScenario's Launch Method

[mrk-han]

Description

As referenced in my original AndroidX Discuss Post and my Stack Overflow post and this new stack overflow post about FragmentScenario, there seems to be a limitation of the ActivityScenario API when using ActivityScenario.launch() method because it only waits for two possible Lifecycle.States RESUMED and DESTROYED.

Claim:
There is a limitation within the launch(Intent startActivityIntent) method of the ActivityScenario API. It waits for the Activity to be Lifecycle.STATE.RESUMED or DESTROYED and if it isn't within 4.5 seconds then it throws this error.

Context:
My application uses an IndexActivity to load a config which instructs the application on certain API calls to make. However, immediately after it loads a DialogActivity and the IndexActivity goes into STOPPED. On accepting terms within the DialogActivity the IndexActivity goes back into RESUMED and then ActivityScenario works properly. With my tests, there was a race condition on whether Espresso could click through the terms within 4.5 seconds to get the IndexActivity to be RESUMED or whether this error would throw before that. It would take major refactoring to enable another Activity to be launched with ActivityScenario so that was not an option.

The Fix
Within public static <A extends Activity> ActivityScenario<A> launch(Intent startActivityIntent) of Activity Scenario, check the logic scenario.waitForActivityToBecomeAnyOf(State.RESUMED, State.DESTROYED);

If you can create your own custom Activity Scenario and adjust this line of code to be something like scenario.waitForActivityToBecomeAnyOf(State.STOPPED, State.DESTROYED); then it will theoretically work for you. You can then use ActivityScenario again to move the Activity into whatever Lifecycle State you want.

OR just use the old https://developer.android.com/reference/androidx/test/rule/ActivityTestRule

TL;DR
This is happening because the Lifecycle.State of your Activity is not either of the two specific lifecycle states ActivityScenario.Launch() waits for, RESUMED or DESTROYED. Your activity is probably in the background of a dialog or another edge-case situation that was not thought about when creating the API.

Full StackTrace for Test here:

10:54:42 V/InstrumentationResultParser: java.lang.AssertionError: Activity never becomes requested state "[RESUMED]" (last lifecycle transition = "STOPPED")
10:54:42 V/InstrumentationResultParser: at androidx.test.core.app.ActivityScenario.waitForActivityToBecomeAnyOf(ActivityScenario.java:228)
10:54:42 V/InstrumentationResultParser: at androidx.test.core.app.ActivityScenario.moveToState(ActivityScenario.java:368)
10:54:42 V/InstrumentationResultParser: at com.myapplication.android.test.HomeTest.launchActivity(HomeTest.java:30)
10:54:42 V/InstrumentationResultParser: at java.lang.reflect.Method.invoke(Native Method)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
10:54:42 V/InstrumentationResultParser: at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
10:54:42 V/InstrumentationResultParser: at androidx.test.internal.runner.junit4.statement.RunBefores.evaluate(RunBefores.java:76)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
10:54:42 V/InstrumentationResultParser: at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:104)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.Suite.runChild(Suite.java:128)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.Suite.runChild(Suite.java:27)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
10:54:42 V/InstrumentationResultParser: at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
10:54:42 V/InstrumentationResultParser: at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
10:54:42 V/InstrumentationResultParser: at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
10:54:42 V/InstrumentationResultParser: at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
10:54:42 V/InstrumentationResultParser: at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:388)
10:54:42 V/InstrumentationResultParser: at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2075)

Steps to Reproduce

1. Start Activity with ActivityScenario
2. Activity has some condition where either terms have to be accepted or waits a few seconds before going into RESUMED state, due to pop-up modal or dialog

Expected Results

It seems this is not a bug but is rather a restriction of the API
But, the expected behavior would be to either allow STOPPED to be waited for within the launch() method, or to allow the user to specify which states they want to wait for.

Actual Results

ActivityScenario wants your activity to be either RESUMED or DESTROYED, and if it isn't after 45000 milliseconds then it throws the error above.

AndroidX Test and Android OS Versions

android {
    defaultConfig {
        // Specifies instrumentation which connects the test package and the application package
        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"

        // The package name of the test app
        testApplicationId 'com.myapplication.android.test'

        // The following argument makes the Android Test Orchestrator run its
        // "pm clear" command after each test invocation. This command ensures
        // that the app's state is completely cleared between tests.
        testInstrumentationRunnerArguments clearPackageData: 'true'
    }

    testOptions {
        animationsDisabled = true
        execution 'ANDROIDX_TEST_ORCHESTRATOR'
    }

    useLibrary 'android.test.runner'
    useLibrary 'android.test.base'
    useLibrary 'android.test.mock'
}

dependencies {
    // Core library
    androidTestImplementation 'androidx.test:core:1.0.0'

    // AndroidJUnitRunner and JUnit Rules
    androidTestImplementation 'androidx.test:runner:1.1.0'
    androidTestImplementation 'androidx.test:rules:1.1.0'
    androidTestUtil 'androidx.test:orchestrator:1.1.0'

    // Assertions
    androidTestImplementation 'androidx.test.ext:junit:1.0.0'
    androidTestImplementation 'androidx.test.ext:truth:1.0.0'
    androidTestImplementation 'com.google.truth:truth:0.42'

    // Espresso dependencies
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.1.0'
    androidTestImplementation 'androidx.test.espresso:espresso-contrib:3.1.0'
    androidTestImplementation 'androidx.test.espresso:espresso-intents:3.1.0'
    androidTestImplementation 'androidx.test.espresso:espresso-accessibility:3.1.0'
    androidTestImplementation 'androidx.test.espresso:espresso-web:3.1.0'
    androidTestImplementation 'androidx.test.espresso.idling:idling-concurrent:3.1.0'

    // The following Espresso dependency can be either "implementation"
    // or "androidTestImplementation", depending on whether you want the
    // dependency to appear on your APK's compile classpath or the test APK
    // classpath.
    androidTestImplementation 'androidx.test.espresso:espresso-idling-resource:3.1.0'

}

Link to a public git repo demonstrating the problem:

None right now.

[yuuki3655]

Thanks for reporting the issue with the detailed explanation. This behavior was by design because moveToState method doesn't work otherwise. However, given the example you explained there are valid use cases certainly. I updated launch method to accept any of steady states and moveToState's javadoc by my latest commit.

[mrk-han]

Thank you @yuuki3655 ! This will be very helpful.

[alexbakker]

This appears to still be an issue in 1.3.0-rc03. Reverting to the deprecated ActivityTestRule works.

It seems the build/test logs for this repository are not public. Is the redirectingActivityShouldBeLaunchable test failing? If not, I'll try to come up with a small demo app and create a new issue.

[christophehenry]

Can this issue be reopened? I am still able to reproduce it with ActivityRule and ActivityScenarioRule on 1.3.0 on several tests. I am even able to reproduce on an absolutely empty AppCompatActivity. The deprecated ActivityTestRule works, though.

Edit: If you need a live example of the reproductibility, here is the repository. The tests pass on the branch 143-passes and fail on the branch 143-fails and the branch 143-fails-empty demontrate how the test fail even on an empty AppCompatActivity.

The relevant test class is LoginActivityTest which tests LoginActivity (that class is totally commented out on the branch 143-fails-empty).

Strangely, this does not reproduce througout all the tests. For instance, SettingsFragmentTest contains a call to onActivity which produces not error either on my computer on my Jenkins CI.

[WarrenFaith]

It also happens with 1.4.0
@yuuki3655 could you please reopen this issue?

[chriswiesner]

yes, also same problem here with 1.4.0

[zsperske]

I have run into this same issue when updating AGP to 7.2.0

[vrnvorona]

Why waitForActivityToBecomeAnyOf is private? it could be useful in tests

[gustavobarbosab]

I had the same problem here, using the AGP 8.2.1...
I could realize that the problem happens when I'm calling the function close() of the scenario.
Here we're using androidx test core version 1.5.0.