Mac_arm64_android run_release_test is 11.11% flaky

[fluttergithubbot]

The post-submit test builder Mac_arm64_android run_release_test had a flaky ratio 11.11% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_android%20run_release_test/2,%201
Commit: 9d59532

Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_android%20run_release_test/2
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_android%20run_release_test/1

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac_arm64_android%20run_release_test

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

[zanderso]

@keyonghan @jmagman Are these flakes, or part of the arm64 mac bringup?

[keyonghan]

Both builds failed with dependency resolving error:

Executing "/opt/s/w/ir/x/w/recipe_cleanup/tmpazbqkvcw/flutter sdk/bin/flutter --suppress-analytics run --release -d ZY223VC74B lib/main.dart" in "/opt/s/w/ir/x/w/recipe_cleanup/tmpazbqkvcw/flutter sdk/dev/integration_tests/ui" with environment {BOT: false, LANG: en_US.UTF-8}
[run_release_test] [STDOUT] run:stdout: Launching lib/main.dart on Moto G 4 in release mode...
[run_release_test] [STDOUT] run:stdout: Running Gradle task 'assembleRelease'...                        
[run_release_test] [STDOUT] run:stderr: 
[run_release_test] [STDOUT] run:stderr: FAILURE: Build failed with an exception.
[run_release_test] [STDOUT] run:stderr: 
[run_release_test] [STDOUT] run:stderr: * What went wrong:
[run_release_test] [STDOUT] run:stderr: Could not determine the dependencies of task ':app:lintVitalRelease'.
[run_release_test] [STDOUT] run:stderr: > Could not resolve all artifacts for configuration ':app:debugCompileClasspath'.
[run_release_test] [STDOUT] run:stderr:    > Did not resolve 'androidx.test:runner:1.2.0' which is part of the dependency lock state
[run_release_test] [STDOUT] run:stderr:    > Did not resolve 'androidx.test:rules:1.2.0' which is part of the dependency lock state
[run_release_test] [STDOUT] run:stderr:    > Did not resolve 'androidx.test:monitor:1.2.0' which is part of the dependency lock state

But this happened only when we first switched the builders from staging to prod. /cc @blasten any insight whether this is some real flake in Gradle dependency or transitional?

[zanderso]

Routing to @blasten to answer the question from @keyonghan

[blasten]

This is very strange.

In https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_android%20run_release_test/2/overview, Gradle is trying to download an artifact that doesn't exist from Google Maven. Gradle is configured to also look artifacts hosted in Maven central, but for some reason it doesn't continue, and stopped with the first try.
I don't know if this is a bug in Gradle, but that should not happen.

In https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_android%20run_release_test/1, it's a different issue although also related to downloading network artifacts.

@keyonghan would you be able to see if the Gradle cache ($HOME/.gradle) isn't being restored with the pre-downloaded packages? I don't know why that isn't the case, but here's how this is solved in Github actions: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#input-parameters-for-the-cache-action

One possibility is that we move tests to GitHub Action, so we can use this caching mechanism. I suspect this isn't a good solution given how much we depend on LUCI.

[keyonghan]

Based on recent runs, there are not any flake in top 100 commits: https://ci.chromium.org/p/flutter/builders/staging/Mac_arm64_android%20run_release_test?limit=100
I believe this is a transient issue.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.