[Bug] Possible intermittent UI test failure - SettingsAboutTest.verifyAboutFirefoxPreview

[mcomella]

In my PR #9607, I received a UI test failure (firebase) in SettingsAboutTest.verifyAboutFirefoxPreview. However, I only touched unit test code and build code – no production app or UI test code – so the UI test results should remain unchanged.

aaronmt briefly looked into this and noted:

Open bug with the intermittent label for now mcomella -- that one is weird, "Test instrumentation process crashed. Check org.mozilla.fenix.ui.SettingsAboutTest#verifyAboutFirefoxPreview.txt for details"

Also from the video tab:

Video is not available for test cases shorter than 0.5 seconds or where Firebase Test Lab records at a low framerate, such as for virtual devices.

---

aaronmt continued investigation:

Oh the log-cat shows it got the crash reporter dialog and quit
I wonder if we can do some kind of graceful shutdown or better crash identification 04-03 11:54:46.113: I/ActivityManager(1663): Force finishing activity ActivityRecord{fb957a6 u0 org.mozilla.fenix.debug/mozilla.components.lib.crash.prompt.CrashReporterActivity t5}
Are there many crash reports from Firebase AVD devices (how do you even identify them?)

┆Issue is synchronized with this Jira Task

[AaronMT]

https://console.firebase.google.com/u/0/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/5253904521782985921

Crash happening, is there any way to associate a report here with full details?

The line before the crash

04-05 20:25:14.272: I/ViewInteraction(8143): Performing 'single click' action on view with text: is "Libraries that we use"

04-05 20:29:14.754: I/ActivityManager(1662): Start proc 9736:org.mozilla.fenix.debug:mozilla.components.lib.crash.CrashHandler/u0a95 for service org.mozilla.fenix.debug/mozilla.components.lib.crash.handler.CrashHandlerService

04-05 20:29:14.980: I/mozac/CrashReporter(9736): Received crash: NativeCodeCrash(minidumpPath=/data/user/0/org.mozilla.fenix.debug/files/mozilla/5l1x879t.default/minidumps/68ecf2dd-4442-24dd-a687-da33c63b7a6f.dmp, minidumpSuccess=false, extrasPath=/data/user/0/org.mozilla.fenix.debug/files/mozilla/5l1x879t.default/minidumps/68ecf2dd-4442-24dd-a687-da33c63b7a6f.extra, isFatal=true, breadcrumbs=[])

04-05 20:29:15.029: I/ActivityManager(1662): Process org.mozilla.fenix.debug (pid 7743) has died: fore TOP

04-05 20:29:14.983: I/ActivityManager(1662): START u0 {flg=0x10018000 cmp=org.mozilla.fenix.debug/mozilla.components.lib.crash.prompt.CrashReporterActivity (has extras)} from uid 10095
04-05 20:29:14.992: I/WindowManager(1662): WIN DEATH: Window{66d3117 u0 org.mozilla.fenix.debug/org.mozilla.fenix.HomeActivity}

[sv-ohorvath]

A more recent crash in this test (9 days ago): https://console.firebase.google.com/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/5054681088551426089/executions/bs.efdb4b92e00b7e2/testcases/2/test-cases

taken from: https://github.com/mozilla-mobile/fenix/runs/693200739

Maybe this reveals an actual bug in the app.

[sv-ohorvath]

I'm seeing this Test Instrumentation crash more often and in other tests as well:
verifyBasicNavigationToolbarFunctionality
https://console.firebase.google.com/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/7287191204756880523/executions/bs.960426304d4a9b/testcases/2/test-cases
https://console.firebase.google.com/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/8567795915680826663/executions/bs.95c241f917dd0fc5/testcases/2/test-cases

threeDotMenuItemsTest
https://console.firebase.google.com/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/8567795915680826663/executions/bs.cfff703900824186/testcases/2/test-cases

[ekager]

Came here to report that threeDotMenuItemsTest crashing as well

[rpappalax]

@AaronMT @sv-ohorvath let's try and re-enable once this has landed later this week: #12048

[rpappalax]

this is a dupe of: #11642