-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
While building out folder we are facing a issue in MAC #16759
Comments
The issue is here:
Presumably this is an M1 mac, hence the installed library is arm64? Why is this building for We should either fix to detect the right CPU type in the build system, or you might need to install the x86-64 version of openssl if this should really produce an x86-64 binary. |
I was looking a bit into this and as far as I can figure out this seems to be an "issue" with gn. It seems like it is only able to return "x86" or "x64" as the host_cpu (which is typically used to set the current_cpu), so my MacBook Pro M1 has the current_cpu set to x64. So, in the build/config/mac/mac_sdk.gni this results in the target_cpu being set to "x86_64". I tried to look at the gn help to see if I could find a clean way to determine if it is a arm64 based device, but I have not been able to find any. As mentioned, installing the x86_64 homebrew and use that to install openssl and pkg-config also works since it then uses x86_64 versions of the libs. Alternatively I noticed it was possible to pass arguments to the build script, so by using this command I was able to get it to build on my M1 Mac.
I also tested by simply overriding the values in the mac_sdk.gni, forcing the mac_target_arch to "arm64", which also worked fine if you do not want to add the args every time. Note: I bumped the deployment target as well to get rid of warnings that the libs was built for 12.0 but being linked to 11.0. It might work without that part, not sure. |
@mspang Any idea what should be happening here on the |
But do you know why it works well on M1 one week ago? |
Not sure, I seem to remember that homebrew was updated at some point during this process, somy setup might have changed from having the the x86_64 variant of openssl to the the arm64 version. To be honest, I do not know why this suddenly broke, but I was not able to go back to older commits, which I know has worked previously. So I strongly assume this is something that changed in my setup and not in the repo. |
FYI,
then I remove .environment folder and bootstrap.sh again, I got this issue |
Sounds very familiar. We had two MacBooks which also reported an issue on pigweed and the solution we ended up with was also to remove the .enviroment folder and re-bootstrap, so maybe there is some changes in the bootstraping process that is now causing issues with Apple Silicon? Actually a20ba1c was also one of the commits that I used to sanity check, since I know it was working earlier, but it did not work for me afterwards. I might not have properly bootstrapped it after having checked it out, hence why it was not working. Did you remove the .enviroment folder when jumping back? |
No, I have two workspaces, but the old commit id does not work also now with below errors. And I am re-bootstrap that now
|
The downloaded gn is for x86_64 right now The old version gn is for arm64 I think that`s the reason |
I think I found the reason, as the below code in pigweed shows, third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/cipd_setup/wrapper.py, on M1 macos, pigweed will force download Rosetta cipd and gn. This is change comes from google/pigweed@f1511cd, anyone can connect to google and get the reason?
|
Good catch! |
I revert all above rosetta code in pigweed and it works now. anyone can connect to google about that?
|
Forwarded this to pigweed team. Tagging @keir for visibility. |
We're working on it. The reason this was changed upstream in pigweed was that not all of their tools have native mac-arm64 builds yet. In the meantime, besides reverting the bootstrap change, there are a couple of solutions (untested, I don't have an M1 device):
|
@rgoliver Is there a roll that could be reverted here or would that cause other issues? |
#16537 was the last change that updated pigweed, it was a couple weeks ago. I checked and reverting that would drop f1511cd change mentioned above, so should fix the issue. That CL provides decoding of tokenized logs for NXP, they can always roll back to build the tool in the short term so not a big issue to drop it. The risk would be that a change went in within the past couple weeks that depended on the pigweed update, hopefully this would be caught by CI though. I did bootstrap and a couple builds locally and they all worked, so I suspect it's OK but CI would be the real test, and then watching for any other breakages on targets not covered by CI. |
Hey everyone, I'm on the Pigweed team and I'm the author of the change that broke things for you 🙂 I have a fix in upstream Pigweed that should solve this problem once it rolls into this repo. To give some additional context: Pigweed didn't have official support for M1 Macs prior to google/pigweed@f1511cd, because we didn't have (and still don't have) a complete mac-arm64 toolchain available yet in CIPD, the package manager that Pigweed uses. For Pigweed projects that use the Pigweed toolchain, things didn't work on M1 Macs at all. So the temporary solution to provide experimental support on M1 Macs was to simply act like Intel Macs and use the mac-amd64 toolchain via Rosetta. This works well for teams using the Pigweed toolchains, but clearly not well if you're using the host toolchain from outside of Pigweed and external native libraries, since you'll end up with this architecture mismatch. Now there's a bootstrap environment variable that controls whether bootstrap pulls the incomplete mac-arm64 toolchain (which is sufficient for Matter) or the complete mac-amd64 toolchain (which most Pigweed projects need to run on M1 Macs right now). Since you are not setting this variable in your bootstrap script, I expect that things will just work again once this change rolls down. |
Awesome, thanks for the detailed explanation. It sounds like we have a few ways to workaround the issue until the fix is included in the SDK, so maybe we should just close this issue for now? @bzbarsky-apple |
I setup a PR to fix it, but seem met another issue while bootstrap [4/247] python3 ../../third_party/pigweed/repo/pw_build/py/pw_build/python_runner.py --gn-root ../../ --current-path ../../third_party/pigweed/repo/pw_unit_test --lockfile protocol_buffer/pip.lock --default-toolchain=//build/toolchain/host:mac_arm64_gcc --current-toolchain=//third_party/pigweed/repo/pw_protobuf_compiler/toolchain:protocol_buffer --capture-output -- ../../third_party/pigweed/repo/pw_protobuf_compiler/py/pw_protobuf_compiler/generate_protos.py --language python --include-file protocol_buffer/gen/third_party/pigweed/repo/pw_unit_test/unit_test_proto.proto_library/includes.txt --compile-dir protocol_buffer/gen/third_party/pigweed/repo/pw_unit_test/unit_test_proto.proto_library/sources --out-dir protocol_buffer/gen/third_party/pigweed/repo/pw_unit_test/unit_test_proto.proto_library/python --sources protocol_buffer/gen/third_party/pigweed/repo/pw_unit_test/unit_test_proto.proto_library/sources/pw_unit_test_proto/unit_test.proto |
Hi @chadnorvell @keir above issue, is caused by the below changes in upstream Pigweed, could you please revert related code? Thanks
|
@xylophone21 Re: the patch you showed in this comment, I'm looking into getting a mac-arm64 build of protoc into CIPD so that this hack isn't necessary. If that can't be done today, I'll revert that change for now to unblock you. |
@chadnorvell you can ref this error log, seems next steps need protoc |
That change is now reverted. This should resolve the |
This issue is fixed by #17247, I think it can be closed |
Problem
With commitID(58cd5fb) we are facing this issue while trying to built out folder in MAC i,e.,
" ld: symbol(s) not found for architecture x86_64"
Steps to reproduce :
PFA log below
Building error.txt
The text was updated successfully, but these errors were encountered: