Errors in creating and maintaining UiAutomation connection in Instrumented tests

[AaronMT]

Intermittently, an early getInstrumentation() call in beforeActivityLaunched for our HomeActivityTestRule.kt will throw a java.lang.RuntimeException with a Error while connecting UiAutomation. This seems to happen more frequent on 1.4.0 and espresso 3.4.0 libraries [2] - just getting a ticket on file here to wrap this call and try again (although I don't know if multiple attempts to get a handle on the running instrumentation will do anything). This is for executing a shell command to set a timeout on long tap.

Stack:

java.lang.RuntimeException: Error while connecting UiAutomation@afc456f[id=-1, flags=0]
at android.app.UiAutomation.connect(UiAutomation.java:259)
at android.app.Instrumentation.getUiAutomation(Instrumentation.java:2176)
at androidx.test.uiautomator.UiDevice.getUiAutomation(UiDevice.java:1129)
at androidx.test.uiautomator.QueryController.<init>(QueryController.java:95)
at androidx.test.uiautomator.UiDevice.<init>(UiDevice.java:109)
at androidx.test.uiautomator.UiDevice.getInstance(UiDevice.java:261)
at org.mozilla.fenix.helpers.HomeActivityTestRuleKt.setLongTapTimeout(HomeActivityTestRule.kt:78)

[1] Firebase Log
[2] Ticket filed as android/android-test#1305

┆Issue is synchronized with this Jira Task

[AndiAJ]

Corresponding intermitent tests:
customTabShareTextTest #25099
privateBrowsingUseAppWhileTabIsCrashedTest #25076
verifyExpandedCollectionItemsTest #25075
verifyContextOpenImageNewTab #25054

[AndiAJ]

Hey @AaronMT , it's still happening.
closeAllTabsTest #25341
openMainMenuSettingsItemTest #25342

[sv-ohorvath]

New cases started to appear:
saveLoginFromPromptTest #25414
addPrivateBrowsingShortcut #25416
customTabsOpenExternalLinkTest #25028
downloadMultipleFileTypesTest #25296

[AaronMT]

I filed an issue on a Google issue tracker, I have no idea what causes this. I suspect it's a particular in androidx.test.uiautomator:uiautomator

https://issuetracker.google.com/u/1/issues/233914320

[sv-ohorvath]

java.lang.IllegalStateException: UiAutomation not connected, UiAutomation in the following issues:
https://github.com/mozilla-mobile/fenix/issues?q=is%3Aissue+is%3Aopen+java.lang.IllegalStateException%3A+UiAutomation+not+connected%2C+UiAutomation

[AaronMT]

FTL team responded back on Slack that a potential fix should be in as of yesterday (06/29), let's keep a track on any reoccurrence and I'll reply back.

[AndiAJ]

► 6/30

java.lang.IllegalStateException: UiAutomation not connected, UiAutomation
• searchSettingsItemsTest

java.lang.RuntimeException: Error while connecting UiAutomation
• verifyMultipleLoginsSelectionsTest
• mainMenuShareButtonTest
• closePrivateTabTest
• emptyTabsTrayViewPrivateBrowsingTest
• openMainMenuAddToCollectionTest

[AndiAJ]

► 7/4

java.lang.RuntimeException: Error while connecting UiAutomation
• deleteTabsDataWithNoOpenTabsTest

[AndiAJ]

► 7/5

java.lang.IllegalStateException: UiAutomation not connected, UiAutomation
• navigateBookmarksFoldersTest
• emailLinkPWATest

[AaronMT]

Firebase Team Update:

Hello Aaron,
Thank you for your patience while this issue was troubleshooted.
The product team have narrowed down the issue to GmsCore, which is encountering issues with virtual images.
An update is scheduled sometime in the next few weeks which should address this issue.
Until the update is rolled out, the Test Lab team suggests that we should monitor the public issue [1].
Please let me know if you have any further questions regarding this issue.
Thank you

[1] https://issuetracker.google.com/issues/233914320

Also in communication with a member on the Firebase Community Slack about a fix too.

[AndiAJ]

► 7/5

java.lang.IllegalStateException: UiAutomation not connected, UiAutomation
• deleteHistoryItemTest
• multipleBookmarkDeletionsTest
• editBookmarkTest

java.lang.RuntimeException: Error while connecting UiAutomation
• verifyAddToFirefoxHome
• verifyUndoSnackBarTest
• openLinkInAppTest
• testCloseDownloadPrompt
• testDownloadCompleteNotification
• editBookmarkTest

[Mugurell]

Saw the issue again in #26299 for verifyEmptyTabTray.

[AaronMT]

Currently waiting on Google to push out an update to Firebase Test Lab to ensure all devices are using newer GMSCore (FLOSS Play Services) version which was the culprit of this problem. Currently in test runs, some devices are on the newer Play Services version (you can tell from the first frame in videos) and some are using the older version which is another bug.

[AaronMT]

FTL Update (09.15)

The fix has finally been pushed to production. The issue should be resolved now, please give it a try.

We've also switched to Pixel2.arm (ARM) running a new GMSCore (Play Services), and have knock on wood yet to see see too. We can re-enable the affected tests and close this out.

[AaronMT]

This was resolved on their end.