Linux customer_testing is 2.00% flaky

[fluttergithubbot]

The post-submit test builder Linux customer_testing had a flaky ratio 2.00% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing/16647,%2016648,%2016646
Commit: e5762ba

Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing/16648
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing/16647
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing/16646
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing/16601

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20customer_testing

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

[goderbauer]

Three of the four flaky builds listed above happened in dev_tools (the fourth one was in macos_ui). @kenzieschmoll can you take a look to see if any action is required to make the dev_tools tests less falky?

[kenzieschmoll]

The inspector one may have been a real failure - there have been some changes made recently around detecting package directories for the inspector.

For the tests that flaked due to eval_integration_test.dart, we should consider skipping this test for the flutter customer tests.

[fluttergithubbot]

[prod pool] flaky ratio for the past (up to) 100 commits between 2023-12-13 and 2023-12-19 is 1.00%. Flaky number: 1; total number: 100.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing/16904
Commit: b167f06
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20customer_testing/16904

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20customer_testing

[goderbauer]

Failure was:

| 04:45 +1217 ~8 -1: /b/s/w/ir/x/t/flutter_customer_testing.flutter_devtools.FGNCBV/tests/packages/devtools_app/test/shared/eval_integration_test.dart: EvalOnDartLibrary asyncEval returns the result of the future completion [E]
|   Evaluation `() {  final result = widgetInspectorService.toObject("inspector-0", "asyncEval-2") as List;  widgetInspectorService.disposeGroup("asyncEval-2");  return result;}()` failed with [ErrorRef id: objects/6, kind: UnhandledException, message: Unhandled exception:
|   type 'Null' is not a subtype of type 'List<dynamic>' in type cast
|   #0      Eval.<anonymous closure> ()
|   #1      Eval ()
|   ] for scope: {widgetInspectorService: objects/2}
|   package:devtools_app_shared/src/service/eval_on_dart_library.dart 502:11  EvalOnDartLibrary.safeEval

@kenzieschmoll Do you know if this was a flake or a real failure? If it was real, we can probably close this?

Edit: From your previous response it sounds like this was an actual flake?

For the tests that flaked due to eval_integration_test.dart, we should consider skipping this test for the flutter customer tests.

Could somebody from devtools look into your suggestion so we can get this flake resolved?

[kenzieschmoll]

This one is likely a flake. Skipping here: flutter/devtools#6999