Run all flutter/engine macOS tests manually on macOS 14.5 bot

[vashworth]

Prerequisites

• Upgrade 1 arm64 and 1 x86 chromium staging bots to macOS 14.5 #148860

Instructions

Update to latest depot_tools by running gclient:

For each macOS engine test, you have to run the sub-builds individually. Using test/orchestrator Linux linux_web_engine as an example:

1. Open https://ci.chromium.org/ui/p/flutter/builders/luci.flutter.prod/Linux%20linux_web_engine
2. Click the top build
3. Ensure build is for main branch (Under the Infra tab, check git_branch under "Input Properties" section)
4. Check what sub-builds use macOS
   • https://github.com/flutter/engine/blob/main/ci/builders/linux_web_engine.json
   • Search for os=Mac
5. Copy sub-build BuildbucketID
   
6. Run led get-build "[insert BuildbucketID]" > properties.json
7. Replace the following in the properties.json:
   • "bucket": "prod.shadow", -> "bucket": "staging.shadow",
   • "shadowed_bucket": "prod" -> "shadowed_bucket": "staging"
   • "service_account": "flutter-prod-builder@chops-service-accounts.iam.gserviceaccount.com" -> "service_account": "flutter-staging-builder@chops-service-accounts.iam.gserviceaccount.com"
   • "value": "luci.flutter.prod" -> "value": "luci.flutter.staging"
8. Run cat properties.json | led edit-recipes-bundle | led launch

❗ File issues for tests that fail and add links to the issues in #148882. This is important so we can track any fixes that need to be cherrypicked to beta/stable and so steps that are steps are not completed before issues are resolved.

[vashworth]

Tracking Progress:

• Mac builder_cache
• Linux mac_android_aot_engine
  • Builds:
    • android_profile
    • android_profile_arm64
    • android_profile_x64
    • android_release
    • android_release_arm64
    • android_release_x64
• Mac mac_clang_tidy
  • Builds:
    • host_debug_clang_tidy
    • ios_debug_sim_clang_tidy
  • Tests:
    • test: lint host_debug 0
    • test: lint host_debug 1
    • test: lint host_debug 2
    • test: lint host_debug 3
    • test: lint ios_debug_sim
• Mac mac_host_engine
  • Builds:
    • host_debug
    • host_profile
    • host_release
    • mac_debug_arm64
    • mac_profile_arm64
    • mac_release_arm64
      • Tracking issue: [Mac-14] Impeller-golden for host_release fails on macOS 14 #149416
      • 🔴 https://ci.chromium.org/b/8746478703122747313
      • 🔴 https://ci.chromium.org/b/8746413109650410993
      • ✅ https://ci.chromium.org/b/8745953029182995153 (after approving golden image)
      • ✅ https://ci.chromium.org/b/8745947837818717841 (after approving golden image)
• Linux mac_clangd
  • Builds:
    • mac_unopt_debug_no_rbe
• Mac mac_unopt
  • Builds:
    • host_debug_unopt
    • ios_debug_unopt_sim
    • host_debug_unopt_arm64
    • ios_debug_unopt_sim_arm64
    • ios_debug_unopt_sim_arm64_extension_safe
    • macos/ios_release_slimpeller
    • macos/ios_debug_unopt_slimpeller
• Mac mac_ios_engine
  • Builds:
    • ios_debug_sim
    • ios_debug_sim_arm64
    • ios_debug
    • ios_profile
    • ios_release
    • ios_debug_sim_extension_safe
    • ios_debug_sim_arm64_extension_safe
    • ios_debug_extension_safe
    • ios_profile_extension_safe
    • ios_release_extension_safe
• Mac impeller-cmake-example (doesn't pass in master so not bothering to test it)
• Linux linux_web_engine
  • Tests:
    • Mac run safari-dart2js-html-engine suite
    • Mac run safari-dart2js-html-html suite
    • Mac run safari-dart2js-html-ui suite
    • Mac run safari-dart2js-canvaskit-canvaskit suite
    • Mac run safari-dart2js-canvaskit-ui suite
    • Mac run safari-fallbacks suite
• Linux local_engine_builds
  • Builds:
    • ios_debug
    • ios_debug_unopt
    • ios_profile
    • ios_release
    • ios_debug_sim
    • ios_debug_sim_arm64
    • android_debug_arm64
    • android_debug_unopt_arm64
    • android_profile_arm64
    • android_release_arm64
    • host_debug
    • host_debug_unopt_arm64
    • host_profile_arm64
    • host_release_arm64
    • host_profile
    • host_release

[vashworth]

Ran and verified all tests pass.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.