Linux_android android_semantics_integration_test is flaky on every commit

[keyonghan]

The flake happens since 0fa15e04acd8db185ab17e5d1d351ecbcbd7b51f

Example build: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8784073433306279057/+/u/run_android_semantics_integration_test/test_stdout

[2023-04-11 15:26:46.173499] [STDOUT] stdout: [+1144 ms] 00:09 �[32m+0�[0m�[31m -1�[0m: AccessibilityBridge TextField TextField has correct Android semantics �[1m�[31m[E]�[0m�[0m
[2023-04-11 15:26:46.173539] [STDOUT] stdout: [        ]   Expected: AndroidSemanticsNode with className: android.widget.EditText with actions: [AndroidSemanticsAction.click, AndroidSemanticsAction.copy, AndroidSemanticsAction.setSelection, AndroidSemanticsAction.setText] with flag isEditable: true with flag isFocusable: true with flag isFocused: true with flag isPassword: false
[2023-04-11 15:26:46.173550] [STDOUT] stdout: [        ]     Actual: AndroidSemanticsNode:<{flags: {isEditable: true, isEnabled: true, isDismissible: false, isCheckable: false, isChecked: false, isFocused: true, isLongClickable: false, isFocusable: true, isPassword: false}, id: 4, text: null, rect: {height: 144, width: 1080, bottom: 128.0, top: 80.0, left: 0.0, right: 360.0}, actions: [131072, 2097152, 16, 128], className: android.widget.EditText, liveRegion: 1, contentDescription: null}>
[2023-04-11 15:26:46.173559] [STDOUT] stdout: [        ]      Which: Expected actions: [AndroidSemanticsAction.click, AndroidSemanticsAction.copy, AndroidSemanticsAction.setSelection, AndroidSemanticsAction.setText]
[2023-04-11 15:26:46.173566] [STDOUT] stdout: [        ]             Actual actions: [AndroidSemanticsAction.clearAccessibilityFocus, AndroidSemanticsAction.click, AndroidSemanticsAction.setSelection, AndroidSemanticsAction.setText]
[2023-04-11 15:26:46.173575] [STDOUT] stdout: [        ]             Unexpected: {AndroidSemanticsAction.clearAccessibilityFocus}
[2023-04-11 15:26:46.173581] [STDOUT] stdout: [        ]             Missing: {AndroidSemanticsAction.copy}
[2023-04-11 15:26:46.180013] [STDOUT] stdout: [   +6 ms]   package:matcher/src/expect/expect.dart 149:31     fail
[2023-04-11 15:26:46.180055] [STDOUT] stdout: [        ]   package:matcher/src/expect/expect.dart 144:3      _expect
[2023-04-11 15:26:46.180063] [STDOUT] stdout: [        ]   package:matcher/src/expect/expect.dart 56:3       expect
[2023-04-11 15:26:46.180072] [STDOUT] stdout: [        ]   test_driver/main_test.dart 153:9                  main.<fn>.<fn>.<fn>
[2023-04-11 15:26:46.180079] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2023-04-11 15:26:46.180088] [STDOUT] stdout: [        ]   package:test_api/src/backend/declarer.dart 215:9  Declarer.test.<fn>.<fn>
[2023-04-11 15:26:46.180095] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2023-04-11 15:26:46.180102] [STDOUT] stdout: [        ]   package:test_api/src/backend/declarer.dart 213:7  Declarer.test.<fn>
[2023-04-11 15:26:46.180126] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2023-04-11 15:26:46.180134] [STDOUT] stdout: [        ]   package:test_api/src/backend/invoker.dart 258:9   Invoker._waitForOutstandingCallbacks.<fn>
[2023-04-11 15:26:46.180156] [STDOUT] stdout: [        ] 00:09 �[32m+0�[0m�[31m -1�[0m: AccessibilityBridge TextField password TextField has correct Android semantics�[0m
[2023-04-11 15:26:46.879596] [STDOUT] stdout: [ +698 ms] 00:10 �[32m+0�[0m�[31m -2�[0m: AccessibilityBridge TextField password TextField has correct Android semantics �[1m�[31m[E]�[0m�[0m
[2023-04-11 15:26:46.879695] [STDOUT] stdout: [        ]   Expected: AndroidSemanticsNode with className: android.widget.EditText with actions: [AndroidSemanticsAction.click, AndroidSemanticsAction.copy, AndroidSemanticsAction.setSelection, AndroidSemanticsAction.setText] with flag isEditable: true with flag isFocusable: true with flag isFocused: true with flag isPassword: true
[2023-04-11 15:26:46.879728] [STDOUT] stdout: [        ]     Actual: AndroidSemanticsNode:<{flags: {isEditable: true, isEnabled: true, isDismissible: false, isCheckable: false, isChecked: false, isFocused: true, isLongClickable: false, isFocusable: true, isPassword: true}, id: 5, text: null, rect: {height: 144, width: 1080, bottom: 592.0, top: 544.0, left: 0.0, right: 360.0}, actions: [131072, 2097152, 16, 128], className: android.widget.EditText, liveRegion: 1, contentDescription: null}>
[2023-04-11 15:26:46.879755] [STDOUT] stdout: [        ]      Which: Expected actions: [AndroidSemanticsAction.click, AndroidSemanticsAction.copy, AndroidSemanticsAction.setSelection, AndroidSemanticsAction.setText]
[2023-04-11 15:26:46.879775] [STDOUT] stdout: [        ]             Actual actions: [AndroidSemanticsAction.clearAccessibilityFocus, AndroidSemanticsAction.click, AndroidSemanticsAction.setSelection, AndroidSemanticsAction.setText]
[2023-04-11 15:26:46.879799] [STDOUT] stdout: [        ]             Unexpected: {AndroidSemanticsAction.clearAccessibilityFocus}
[2023-04-11 15:26:46.879867] [STDOUT] stdout: [        ]             Missing: {AndroidSemanticsAction.copy}
[2023-04-11 15:26:46.880792] [STDOUT] stdout: [        ]   package:matcher/src/expect/expect.dart 149:31     fail
[2023-04-11 15:26:46.880859] [STDOUT] stdout: [        ]   package:matcher/src/expect/expect.dart 144:3      _expect
[2023-04-11 15:26:46.880895] [STDOUT] stdout: [        ]   package:matcher/src/expect/expect.dart 56:3       expect
[2023-04-11 15:26:46.880979] [STDOUT] stdout: [        ]   test_driver/main_test.dart 224:9                  main.<fn>.<fn>.<fn>
[2023-04-11 15:26:46.881011] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2023-04-11 15:26:46.881031] [STDOUT] stdout: [        ]   package:test_api/src/backend/declarer.dart 215:9  Declarer.test.<fn>.<fn>
[2023-04-11 15:26:46.881090] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2023-04-11 15:26:46.881158] [STDOUT] stdout: [        ]   package:test_api/src/backend/declarer.dart 213:7  Declarer.test.<fn>
[2023-04-11 15:26:46.881207] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2023-04-11 15:26:46.881316] [STDOUT] stdout: [        ]   package:test_api/src/backend/invoker.dart 258:9   Invoker._waitForOutstandingCallbacks.<fn>
[2023-04-11 15:26:46.881486] [STDOUT] stdout: [        ] 00:10 �[32m+0�[0m�[31m -2�[0m: AccessibilityBridge TextField (tearDownAll)�[0m
[2023-04-11 15:26:46.970480] [STDOUT] stdout: [  +88 ms] 00:10 �[32m+0�[0m�[31m -2�[0m: AccessibilityBridge SelectionControls (setUpAll)�[0m
[2023-04-11 15:26:47.077444] [STDOUT] stdout: [ +106 ms] I/art     (12833): Do partial code cache collection, code=13KB, data=31KB
[2023-04-11 15:26:47.077524] [STDOUT] stdout: [        ] I/art     (12833): After code cache collection, code=13KB, data=31KB

[reidbaker]

https://ci.chromium.org/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test?limit=200

5 failures in the last 200 runs. Not sure if at that rate if is should be marked as flakey or not.

[chunhtai]

I think the the > 2% is considered flaky

@keyonghan Is the bot still posting the weekly update?

[keyonghan]

I think the the > 2% is considered flaky

@keyonghan Is the bot still posting the weekly update?

It is. @reidbaker F.Y.I. here is the wiki about our flakiness reduction workflow: https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness

[fluttergithubbot]

[staging pool] current flaky ratio for the past (up to) 100 commits is 25.00%. Flaky number: 25; total number: 100.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4144
Commit: 6b6c4bd
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4144
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4138
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4137
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4136
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4131
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4124
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4123
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4122
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4120
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4119
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4114
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4105
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4103
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4098
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4097
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4087
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4072
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4069
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4067
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4063
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4054
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4053
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4050
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4049
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4048
https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_android%20android_semantics_integration_test/4046

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_android%20android_semantics_integration_test