RBE refactor - build fails to copy MacOSX SDK due to Too many levels of symbolic links

[keyonghan]

RBE refactoring is WIP: flutter/engine#49416.

Linux mac_unopt fails when copying MacOSX SDK to CAS: https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20Engine%20Drone/606402/overview

'/Volumes/Work/s/w/ir/x/w/recipe_cleanup/out-cas-directorysxzsrkv_/host_debug_unopt_arm64/gen/SDKs/MacOSX13.3.sdk/System/Library/Frameworks/Ruby.framework/Versions/2.6/Headers/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby/ruby', "[Errno 62] Too many levels of symbolic links:

[keyonghan]

src/out/host_debug_unopt_arm64/gen/SDKs/MacOSX13.3.sdk is a symlink pointing to osx_sdk/xcode_14e300c/XCode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk.

Too many levels of symbolic links happens due to some defult symlinks related to ruby from the xcode CIPD pacakge.

[keyonghan]

Have tried a couple of LED runs based on the drone build:

Failed:

• Attempt1: remove symlink, copy target to src/out, copy directly from xcode/sdk (self.m.path['cache'].join('osx_sdk', 'xcode_14e300c', 'XCode.app', 'Contents', 'Developer', 'Platforms', 'MacOSX.platform', 'Developer', 'SDKs', 'MacOSX13.3.sdk') to build_dir.join('gen', 'SDKs', 'MacOSX13.3.sdk') - Too many levels of symbolic links
• Attempt2: remove symlink, copy target to src/out, zip xcode/sdk - Not all files were readable
• Attempt3: remove symlink, copy target to src/out, recreate symlink in src/out pointing to xcode/sdk - infra error when uploaded to cas
• Attempt4: copy target to src/out with symlinks=True - infra error when uploaded to cas

Succeeded:

• Attempt5: remove symlink, copy target to src/out, upload without MacOS_SDK - Pass

[keyonghan]

Following up with the above Attempt5. By removing symlink and upload to CAS without MacOS_SDK, it also passes on parent build:
Linux mac_unopt
Mac mac_ios_engine
Mac mac_host_engine

Since this affects only the digest uploaded to CAS, it would not affect artifacts uploaded to GCS and then should not affect Framework side.

[keyonghan]

To make it work, I have to manually remove the symlinks of MacOS SDK in three locations:

src/out/`build name`/gen/SDKs - host_debug_unopt & host_debug_unopt_arm64
src/out/`build name`/clang_x64/gen/SDKs - ios_debug_sim
src/out/`build name`/clang_arm64/gen/SDKs - ios_debug_sim_arm64_extension_safe

@zanderso Are these three locations enough? Is there any way to auto retrieve these three locations, I mean clang_x64/clang_arm64?

[zanderso]

I suspect those locations are enough. As for clang_x64 and clang_arm64, there isn't any way to programmatically retrieve those locations from somewhere that I can think of off the top of my head, but those are the standard names for the output directory for targets that are cross-built during our GN build.

You might also walk the tree looking for paths that end in .../gen/SDKs/, and assume that the contents is the symlinks that need to be removed.

[keyonghan]

https://flutter-review.googlesource.com/c/recipes/+/53803 is a potential fix. But a better approach to move the MacOS SDK to a different location works. Closing.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.