Linux web_long_running_tests_5_5 is 2.02% flaky

[fluttergithubbot]

The post-submit test builder Linux web_long_running_tests_5_5 had a flaky ratio 2.02% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15903,%2015902
Commit: 0ee2592

Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15903
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15902
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15852
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15851
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15850

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_5_5

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

[yjbanov]

Not sure what's going on yet, and whether this is related to #143834.

[yjbanov]

Actually, most of the failures are related to #128382 (see this log file). The tests complete OK, but they complain about "Command did not write the response file but expected response file written". Reassigning to @linchen2chris for initial investigation.

[yjbanov]

Looks like I cannot assign to @linchen2chris. So assigning to the first reviewer listed in the PR. @cbracken you drew the short straw :)

[cbracken]

Unassigning since this is one of the web-tests and the issue seems to be related to flutter driver, which which I'm not familiar. Looking at the PR I suspect I was the second required approver. Do we just need someone to revert that patch?

[cbracken]

Looking quickly at the output and code, it looks like:

1. We synchronously delete build/integration_response_data.json
2. We run whatever web test this is
3. We synchronously check for the presence of build/integration_response_data.json and fail to find it causing a failure.

That logic seems correct to me. @yjbanov what makes you think the file is definitely being written (and written on time)?

[fluttergithubbot]

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-03 and 2024-04-09 is 3.19%. Flaky number: 3; total number: 94.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16026
Commit: 7971b35
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16026
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15991
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/15987

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_5_5

[yjbanov]

I actually don't know if it's being written. I also don't know what this file is and why it needs to be written. For now, I'm inclined to revert the change that expects it.

[fluttergithubbot]

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-08 and 2024-04-16 is 2.04%. Flaky number: 2; total number: 98.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16066
Commit: 74b65d4
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16066
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16058

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_5_5

[keyonghan]

Just came across the dashboard and noticed the flaky percentage (11 out recent 50 runs are flaky) continues high: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16274/infra

[yjbanov]

@keyonghan Yes, all the "web_long_running_tests" failures seem to have the same cause.

[fluttergithubbot]

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-18 and 2024-04-23 is 8.25%. Flaky number: 8; total number: 97.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16281,%2016282
Commit: 89a4ffa
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16282
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16281
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16274
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16273
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16272
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16263
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16262
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16257
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16237
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16223
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16206
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16204

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_5_5

[fluttergithubbot]

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-23 and 2024-04-30 is 3.03%. Flaky number: 3; total number: 99.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16336
Commit: 9751d4d
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16336
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16330
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16298

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_5_5

[yjbanov]

This may be the culprit, which I think is what @eyebrowsoffire already suspected:

[0501/183306.597435:INFO:CONSOLE(5)] "Injecting <script> tag. Using callback.", source: http://localhost:35733/flutter_bootstrap.js (5)
[0501/183306.726152:INFO:CONSOLE(168)] "Uncaught Error: Mismatched anonymous define() module: () => CanvasKitInit
http://requirejs.org/docs/errors.html#mismatch", source: http://localhost:35733/require.js (168)

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.