GitHub "Android CI" is unreliable.

[mnalis]

Summary

"Android CI" github workflow fails in majority of cases (11 out of 12 cases for me) with errors like java.lang.IllegalStateException: init() must be called prior to using this method. eg run 9 or run 10 or run 11 until it finally succeeds (without any code changes!) in run 12.
(I've even added [workflow_dispatch] trigger so it can be triggered manually without any repo changes)

• Note that is in latest master (based upon f4e8de7) after fixes in Reduce minsdk back 21 #5236, and Fix failing FilePicker unit test #5237 and that the repository content is the same in cases when it works, and in cases when it fails.

• some earlier runs like run 3 seem to fail with slightly different error androidx.test.espresso.base.RootViewPicker$RootViewWithoutFocusException: Waited for the root of the view hierarchy to have window focus and not request layout for 10 seconds. If you specified a non default root matcher, it may be picking a root that never takes focus. , but those did not have nearby: make sure places is initialized #5241 included yet.

It seems to be a race condition triggered depending on the GitHub servers load or something. The solution would probably be trying to idle for some time while re-checking if the testing environment has become usable, before proceeding with the tests.

Steps to reproduce

1. fork the repository
2. push some data to the fork
3. notice that "Android CI" github workflow is run, but fails majority of the times

Expected behaviour

"Android CI" github workflow should succeed reliably and reproducibly (assuming there are no bugs in the code, of course!). In any case, its behaviour should be deterministic - for same codebase, it should either always succeed or always fail.

Actual behaviour

"Android CI" github workflow fails with many failed test:

fr.free.nrw.commons.MainActivityTest > testExplore[test(AVD) - 7.0] FAILED 
	java.lang.IllegalStateException: init() must be called prior to using this method.
	at androidx.test.espresso.intent.Checks.checkState(Checks.java:70)

or sometimes

fr.free.nrw.commons.AboutActivityTest > testLaunchFacebook[test(AVD) - 7.0] FAILED 
	androidx.test.espresso.base.RootViewPicker$RootViewWithoutFocusException: Waited for the root of the view hierarchy to have window focus and not request layout for 10 seconds. If you specified a non default root matcher, it may be picking a root that never takes focus. Root:
	Root{application-window-token=android.view.ViewRootImpl$W@be181a8, window-token=android.view.ViewRootImpl$W@be181a8, has-window-focus=false, layout-params-type=1, layout-params-string=WM.LayoutParams{(0,0)(fillxfill) sim=#100 ty=1 fl=#81010100 pfl=0x20000 wanim=0x1030465 needsMenuKey=2}, decor-view-string=DecorView{id=-1, visibility=VISIBLE, width=320, height=640, has-focus=true, has-focusable=true, has-window-focus=false, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=WM.LayoutParams{(0,0)(fillxfill) sim=#100 ty=1 fl=#81010100 pfl=0x20000 wanim=0x1030465 needsMenuKey=2}, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2}}

Device name

No response

Android version

n/a

Commons app version

n/a

Device logs

No response

Screen-shots

No response

Would you like to work on the issue?

None

[mnalis]

There is closed issue #4958 that mentions very similar problem - it seems seems the attempted workaround employed there was to not run tests in some cases.

[sivaraam]

2. Push some data to the fork

Could you elaborate about this one? Is there any chnge that might affect the CI run? 🤔

[mnalis]

Push some data to the fork
Could you elaborate about this one? Is there any chnge that might affect the CI run? 🤔

No, there shouldn't be any change that would affect those failures. In detail:

• same problem arises even on initial fork (run 1) when everything is same (by definition)
• the reason why some push (or PR) is needed is because android.yml specifies that CI runs only on on: [push, pull_request], so otherwise I couldn't force it to run multiple times to rule-out one-off failures
• So I've changed that line to say on: [push, pull_request, workflow_dispatch] so there is manual button in GitHub Actions tab, so I can retry it as many times as needed. (As it is generally useful, and not just for this test, I've made an one-liner PR for that: Allow Android CI build to be run manually on demand #5248)
• then I've just after each failed build clicked that button to run new build again without any repository changes, until it finally succeeded after several retries.

[rohit9625]

The CI fails only on our forked repo not on the main repo. So, I don't think this is an issue at all.
What do you think Mr. @sivaraam?

[mnalis]

The CI fails only on our forked repo not on the main repo. So, I don't think this is an issue at all.

Why wouldn't that be an issue?! There is only one original repo, and more then 1200 forked ones. Certainly that is all the more reasons to fix it, then if that was just one broken repo and 1200 working ones 🥲

It still fails for me: https://github.com/mnalis/apps-android-commons/actions/runs/7646438188/job/20835243644

Perhaps those java.lang.IllegalStateException: init() must be called prior to using this method are related to this:
https://stackoverflow.com/questions/33624802/android-espresso-intents-test-randomly-fail-with-init-must-be-called-prior-t

[rohit9625]

Well, I have faced this error before and just now. See here

I thought that it was happening because my repo was forked from the main and my workflow file was not configured to run on my repo. And if I am wrong then the issue needs to be fixed.

Sorry, I misunderstood the issue.

[sivaraam]

I thought that it was happening because my repo was forked from the main and my workflow file was not configured to run on my repo. And if I am wrong then the issue needs to be fixed.

The workflow file should be generic of the repo, Rohit. The workflow file doesn't need any tweaks when a repo has been forked, ideally.

The CI is actually failing our repo itself [ ref ]. We indeed need to check the same and fix it.

[rohit9625]

Yes, you are right. The problem is with the instrumentation tests. I am investigating why is this failling.

[nicolas-raoul]

Do coverage checks run successfully on aonyone's environment?

[rohit9625]

I want to run some tests but this code:-
private fun getTestUsername(): String { val username = BuildConfig.TEST_USERNAME if (StringUtils.isEmpty(username) || username == "null") { throw NotImplementedError("Configure your beta account's username") } else return username }

Throwing the exception:-
Configure your beta account's username

Can anyone help me with the configuration of the username?

[sivaraam]

Do coverage checks run successfully on aonyone's environment?

I suppose not. I think issue mentioned in #5061 still exists.

Can anyone help me with the configuration of the username?

Could you let us know how you ran the test?

[RitikaPahwa4444]

Hi @rohit9625, please add these properties to your local.properties file (add appropriate values):

• TEST_USER_NAME=<your_beta_account_username>
• TEST_USER_PASSWORD=<your_password>

Also, please avoid doing a git add . if local.properties shows up as one of the changes, because these are local changes, plus passwords should not be committed.

Hope this helps! :)

[rohit9625]

Thank you @RitikaPahwa4444
For providing these details :)

[rohit9625]

New day, new errors 😅

Today I tried building with the command gradlew.bat build but instead of the previous one I am getting a new error as follows:-

Task :app:kaptGenerateStubsProdDebugKotlin FAILED.doPrivileged(AccessController.java:400)

FAILURE: Build completed with 4 failures..ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)

1: Task failed with an exception.

• What went wrong:% EXECUTING [34s]
  Execution failed for task ':app:kaptGenerateStubsBetaDebugKotlin'.

A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
Internal compiler error. See log for more details

• Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.

• What went wrong:
  Execution failed for task ':app:kaptGenerateStubsBetaReleaseKotlin'.

A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
Internal compiler error. See log for more details

• Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.

• What went wrong:
  Execution failed for task ':app:kaptGenerateStubsProdReleaseKotlin'.

A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
Internal compiler error. See log for more details

• Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
==============================================================================

4: Task failed with an exception.

• What went wrong:
  Execution failed for task ':app:kaptGenerateStubsProdDebugKotlin'.

A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
Internal compiler error. See log for more details

• Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
==============================================================================

• Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/8.1.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 37s
162 actionable tasks: 6 executed, 156 up-to-date
PS D:\Git_Kraken\apps-android-commons>

Does Someone have any idea about this?