Run all flutter/flutter macOS tests manually on macOS 14.5 bot

[vashworth]

Prerequisites

• Upgrade 1 arm64 and 1 x86 chromium staging bots to macOS 14.5 #148860
• Upgrade 1 arm64 and 1 x86 devicelab staging bots to macOS 14.5 #148859

Instructions

Update to latest depot_tools by running gclient:

To run manual led test on a Mac-14 staging bot:

led get-builder "luci.flutter.staging:Mac channels_integration_test" | led edit -d os=Mac-14 | led edit-recipes-bundle | led launch

To run on a specific bot:

led get-builder "luci.flutter.staging:Mac channels_integration_test" | led edit -d id=[insert bot id here] | led edit -d pool=luci.flutter.staging | led edit -d cpu= | led edit -d os= | led edit -d device_type= | led edit -d mac_model= | led edit-recipes-bundle | led launch

❗ File issues for tests that fail and add links to the issues in #148882. This is important so we can track any fixes that need to be cherrypicked to beta/stable and so steps that are steps are not completed before issues are resolved.

[vashworth]

Tracking Progress:

• Mac_ios animated_advanced_blend_perf_ios__timeline_summary
• Mac_ios draw_vertices_perf_ios__timeline_summary
• Mac_ios draw_atlas_perf_ios__timeline_summary
• Mac_ios static_path_tessellation_perf_ios__timeline_summary
• Mac_ios dynamic_path_tessellation_perf_ios__timeline_summary
• Mac_benchmark animated_complex_opacity_perf_macos__e2e_summary
• Mac_benchmark basic_material_app_macos__compile
• Mac build_ios_framework_module_test
• Mac_arm64 build_ios_framework_module_test
• Mac_arm64 build_tests_1_4
• Mac_arm64 build_tests_2_4
• Mac_benchmark complex_layout_macos__start_up
• Mac_benchmark complex_layout_scroll_perf_macos__timeline_summary
  • Tracking issue: [Mac-14] Testing flutter run on macOS device hangs and times out #149268
  • 🔴 https://ci.chromium.org/b/8746679575520421601
  • 🔴 https://ci.chromium.org/b/8746594042481190481
  • ✅ https://ci.chromium.org/b/8746590520731858241
  • 🔴 https://ci.chromium.org/b/8746590168309555953
  • ✅ https://ci.chromium.org/b/8745960688870050801 (tested after fix landed)
  • ✅ https://ci.chromium.org/b/8745960287981126145 (tested after fix landed)
• Mac_x64 build_tests_1_4
• Mac_x64 build_tests_2_4
• Mac_x64 build_tests_3_4
• Mac_x64 build_tests_4_4
• Mac_arm64 build_tests_3_4
• Mac_arm64 build_tests_4_4
• Mac customer_testing
• Mac dart_plugin_registry_test
• Mac flavors_test_macos
  • Tracking issue: [Mac-14] Testing flutter run on macOS device hangs and times out #149268
  • 🔴 https://ci.chromium.org/b/8746675799351081201
  • 🔴 https://ci.chromium.org/b/8746590482368847569
  • ✅ https://ci.chromium.org/b/8745955743702474385 (tested after fix landed)
  • ✅ https://ci.chromium.org/b/8745955720735395393 (tested after fix landed)
• Mac_benchmark flutter_gallery_macos__compile
• Mac_arm64 flutter_packaging_test - Can't be run on experimental runs (aka via led)
• Mac_arm64 framework_tests_misc
• Mac_ios animation_with_microtasks_perf_ios__timeline_summary
• Mac_ios backdrop_filter_perf_ios__timeline_summary
• Mac_arm64_ios basic_material_app_ios__compile
• Mac_ios channels_integration_test_ios
• Mac_ios complex_layout_ios__start_up
• Mac_arm64_ios flutter_gallery_ios__compile
• Mac_arm64_ios hello_world_ios__compile
• Mac_arm64_ios integration_test_test_ios
• Mac_ios complex_layout_scroll_perf_ios__timeline_summary
• Mac_ios complex_layout_scroll_perf_bad_ios__timeline_summary
• Mac_ios color_filter_and_fade_perf_ios__e2e_summary
• Mac_ios imagefiltered_transform_animation_perf_ios__timeline_summary
• Mac_arm64_ios hot_mode_dev_cycle_ios__benchmark
• Mac_ios external_ui_integration_test_ios
• Mac_ios route_test_ios
• Mac_ios flavors_test_ios
• Mac_ios flutter_gallery_ios__start_up
• Mac flutter_packaging_test - Can't be run on experimental runs (aka via led)
• Mac_benchmark flutter_view_macos__start_up
• Mac framework_tests_libraries
• Mac framework_tests_impeller
• Mac_x64 framework_tests_misc
• Mac framework_tests_widgets
• Mac gradle_plugin_bundle_test
• Mac_benchmark hello_world_macos__compile
• Mac_arm64 module_test_ios
• Mac_arm64 plugin_lint_mac
• Mac_arm64 tool_host_cross_arch_tests
• Mac_arm64 tool_tests_commands
• Mac_ios flutter_view_ios__start_up
• Mac_x64_ios integration_test_test_ios
• Mac_ios integration_ui_ios_driver
• Mac_ios integration_ui_ios_frame_number
• Mac_ios integration_ui_ios_keyboard_resize
• Mac_ios integration_ui_ios_textfield
• Mac_ios ios_defines_test
• Mac_ios ios_platform_view_tests
• Mac_arm64 verify_binaries_codesigned - Can't be run on master branch
• Mac_arm64 hot_mode_dev_cycle_macos_target__benchmark
• Mac_arm64 ios_app_with_extensions_test
• Mac_arm64 macos_chrome_dev_mode
• Mac_arm64 native_ui_tests_macos
• Mac_arm64 run_debug_test_macos
• Mac_arm64 run_release_test_macos
• Mac_ios large_image_changer_perf_ios
• Mac_ios microbenchmarks_ios
• Mac_ios native_assets_ios
• Mac_ios native_platform_view_ui_tests_ios
• Mac integration_ui_test_test_macos
  • Tracking issue: [Mac-14] Testing flutter run on macOS device hangs and times out #149268
  • 🔴 https://ci.chromium.org/b/8746667651581012305
  • ✅ https://ci.chromium.org/b/8746575504199452161
  • ✅ https://ci.chromium.org/b/8745959936944307153 (tested after fix landed)
  • ✅ https://ci.chromium.org/b/8745959679628220113 (tested after fix landed)
• Mac module_custom_host_app_name_test
• Mac module_host_with_custom_build_test
• Mac module_test
• Mac_benchmark platform_view_macos__start_up
• Mac platform_channel_sample_test_macos
• Mac plugin_dependencies_test
• Mac_x64 plugin_lint_mac
• Mac plugin_test
• Mac_x64 tool_host_cross_arch_tests
• Mac_x64 tool_tests_commands
• Mac_x64 verify_binaries_codesigned - Can't be run on master branch
• Mac_ios new_gallery_ios__transition_perf
• Mac_ios new_gallery_skia_ios__transition_perf
• Mac_ios platform_channel_sample_test_ios
• Mac_ios platform_channel_sample_test_swift
• Mac plugin_test_ios
• Mac plugin_test_macos
  • Tracking issue: [Mac-14] macOS xcodebuild test fails with "Test runner never began executing tests after launching" #149264
  • 🔴 https://ci.chromium.org/b/8746664489014373665
  • 🔴 https://ci.chromium.org/b/8746586211190654801
  • 🔴 https://ci.chromium.org/b/8746583085127677633
  • ✅ https://ci.chromium.org/b/8745959249396139041 (tested after fix landed)
  • ✅ https://ci.chromium.org/b/8745958481319350289 (tested after fix landed)
• Mac tool_integration_tests_1_4
• Mac tool_integration_tests_2_4
• Mac_ios platform_channels_benchmarks_ios
• Mac_ios platform_interaction_test_ios
• Mac_ios platform_view_ios__start_up
• Mac_ios platform_views_scroll_perf_ios__timeline_summary
• Mac tool_integration_tests_3_4
• Mac tool_integration_tests_4_4
  • Tracking issue: [Mac-14] Testing flutter run on macOS device hangs and times out #149268
  • 🔴 https://ci.chromium.org/b/8746662000454391073
  • ✅ https://ci.chromium.org/b/8745955202576273201 (tested after fix landed)
  • ✅ https://ci.chromium.org/b/8745955170309724209 (tested after fix landed)
• Mac tool_tests_general
• Mac web_tool_tests
• Mac_x64 hot_mode_dev_cycle_macos_target__benchmark
• Mac_x64 ios_app_with_extensions_test
• Mac_x64 macos_chrome_dev_mode
• Mac_ios platform_views_scroll_perf_ad_banners__timeline_summary
• Mac_ios platform_views_scroll_perf_non_intersecting_impeller_ios__timeline_summary
• Mac_ios post_backdrop_filter_perf_ios__timeline_summary
• Mac_ios simple_animation_perf_ios
• Mac_x64 hot_mode_dev_cycle_ios_simulator
• Mac_ios wide_gamut_ios
• Mac_x64_ios hot_mode_dev_cycle_ios__benchmark
• Mac_ios fullscreen_textfield_perf_ios__e2e_summary
• Mac_ios very_long_picture_scrolling_perf_ios__e2e_summary
• Mac native_assets_ios_simulator
• Mac channels_integration_test
• Mac run_debug_test_macos
  • Tracking issue: [Mac-14] Testing flutter run on macOS device hangs and times out #149268
  • 🔴 https://ci.chromium.org/b/8746658578561332369
  • ✅ https://ci.chromium.org/b/8745955661051629873 (tested after fix landed)
  • ✅ https://ci.chromium.org/b/8745955645622125649 (tested after fix landed)
• Mac_benchmark flutter_tool_startup__macos
• Mac_ios tiles_scroll_perf_ios__timeline_summary
• Mac_ios animated_blur_backdrop_filter_perf_ios__timeline_summary
• Mac_ios draw_points_perf_ios__timeline_summary
• Mac_ios spell_check_test
• Mac_build_test flutter_gallery__transition_perf_e2e_ios - Unable to run via led due to it using same identifier for the build so it doesn't re-build properly
• Mac_x64 native_ui_tests_macos
• Mac_arm64_android integration_test_test
• Mac_android hot_mode_dev_cycle__benchmark
• Mac_android microbenchmarks
• Mac_arm64_android hello_world_android__compile
• Mac_arm64_android run_debug_test_android
• Mac_arm64_android run_release_test
• Mac_pixel_7pro entrypoint_dart_registrant - ran with a msm8952 device
• Mac_pixel_7pro hello_world_android__compile
• Mac_pixel_7pro integration_test_test
• Mac_pixel_7pro integration_ui_frame_number - ran with a msm8952 device
• Mac_pixel_7pro native_assets_android - ran with a msm8952 device
• Mac_pixel_7pro run_debug_test_android
• Mac_pixel_7pro run_release_test

[vashworth]

Ran and verified all tests that I could run pass

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.