Intermittent UI test failure - SmokeTest.editCustomSearchEngineTest

[sv-ohorvath]

Firebase Test Run:

https://console.firebase.google.com/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/8266266295243895427/executions/bs.3645e7ee27e73c92/testcases/1/test-cases

Stacktrace:

java.lang.IllegalArgumentException: Can't scroll to index 5, it is out of bounds [0, 0) at androidx.compose.foundation.lazy.LazySemanticsKt$lazyListSemantics$1$3.invoke(LazySemantics.kt:78) at androidx.compose.foundation.lazy.LazySemanticsKt$lazyListSemantics$1$3.invoke(LazySemantics.kt:77) at androidx.compose.ui.test.ActionsKt$performScrollToIndex$2.invoke(Actions.kt:145) at androidx.compose.ui.test.ActionsKt$performScrollToIndex$2.invoke(Actions.kt:144) at androidx.compose.ui.test.junit4.AndroidSynchronization_androidKt$sam$java_util_concurrent_Callable$0.call(Unknown Source:2) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:458) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at android.app.Instrumentation$SyncRunnable.run(Instrumentation.java:2163) at android.os.Handler.handleCallback(Handler.java:873) at android.os.Handler.dispatchMessage(Handler.java:99) at android.os.Looper.loop(Looper.java:193) at android.app.ActivityThread.main(ActivityThread.java:6669) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

Build:

Main 9/28

┆Issue is synchronized with this Jira Task

[AndiAJ]

Flaky 1x on 9/29 Main Firebase link

[AndiAJ]

Fixed with #21591

[Archaeopteryx]

Still observed for 0d9e2b3 (log).

[sv-ohorvath]

Failed 2/2 runs on 10/1: logs

new stacktrace:
java.lang.AssertionError at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertTrue(Assert.java:53) at org.mozilla.fenix.ui.robots.SettingsSubMenuSearchRobot.saveNewSearchEngine(SettingsSubMenuSearchRobot.kt:61) at org.mozilla.fenix.ui.SmokeTest$editCustomSearchEngineTest$4.invoke(SmokeTest.kt:565) at org.mozilla.fenix.ui.SmokeTest$editCustomSearchEngineTest$4.invoke(SmokeTest.kt:561) at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openSearchSubMenu(SettingsRobot.kt:128) at org.mozilla.fenix.ui.SmokeTest.editCustomSearchEngineTest(SmokeTest.kt:561)

[AndiAJ]

Fixed with #21803

[Archaeopteryx]

Still an issue.

[AaronMT]

@AndiAJ Can you take a look at this one tomorrow?

java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at org.mozilla.fenix.ui.robots.SettingsSubMenuSearchRobot.saveNewSearchEngine(SettingsSubMenuSearchRobot.kt:60)
at org.mozilla.fenix.ui.SmokeTest$editCustomSearchEngineTest$4.invoke(SmokeTest.kt:563)
at org.mozilla.fenix.ui.SmokeTest$editCustomSearchEngineTest$4.invoke(SmokeTest.kt:559)
at org.mozilla.fenix.ui.robots.SettingsRobot$Transition.openSearchSubMenu(SettingsRobot.kt:128)
at org.mozilla.fenix.ui.SmokeTest.editCustomSearchEngineTest(SmokeTest.kt:559)

[AndiAJ]

Disabled it in #21881
It looks like the search string to use is set only partially leading to an error when trying to save the custom search engine (saveNewSearchEngine())

[AndiAJ]

The fix from #21884 is live for almost a day and seems to working properly.
I'll go ahead and close this issue.