Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apple Silicon support #928

Closed
darthdeus opened this issue Nov 25, 2020 · 35 comments · Fixed by #1043
Closed

Apple Silicon support #928

darthdeus opened this issue Nov 25, 2020 · 35 comments · Fixed by #1043
Labels
C-Bug An unexpected or incorrect behavior O-MacOS Specific to the MacOS (Apple) desktop operating system

Comments

@darthdeus
Copy link
Contributor

Bevy version

01ba7c4

Operating system & version

macOS Big Sur 11.0.1

What you did

Ran cargo run --example breakout

What you expected to happen

I expected the example to compile and run.

What actually happened

The example failed to compile with the following error

error: failed to run custom build command for `bevy-glsl-to-spirv v0.2.1`

Caused by:
  process didn't exit successfully: `/Users/darth/projects/bevy/target/debug/build/bevy-glsl-to-spirv-fa67d3095c93334b/build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at 'Missing target support aarch64-apple-darwin', /Users/darth/.cargo/registry/src/github.com-1ecc6299db9ec823/bevy-glsl-to-spirv-0.2.1/build/build.rs:24:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
@cart cart added C-Bug An unexpected or incorrect behavior O-MacOS Specific to the MacOS (Apple) desktop operating system labels Nov 25, 2020
@cart
Copy link
Member

cart commented Nov 25, 2020

Good to know. We might be able to work around this by using shaderc on the aarch64-app-darwin target (which we already do for ios).

@mockersf
Copy link
Member

which can be done around here if someone with a new Mac wants to try (adding the arch at the same place of the iOS target is checked for, and also probably a few changes in the crate too)

@joshuajbouw
Copy link
Member

joshuajbouw commented Nov 27, 2020

@cart 11.0 Big Sur can be added to the CI as the preview had been added but, I'm not entirely too sure if it actually is compiling to the ARM target or not. It should be able to else ARM chip Apple would've had nothing at launch haha.

Would be a good idea to add that target to shaderc and trying to build an ARM example for Big Sur as a new CI much like Android.

@joshuajbouw
Copy link
Member

joshuajbouw commented Nov 27, 2020

Update: https://blog.rust-lang.org/2020/11/27/Rustup-1.23.0.html <- M1 support just released it looks like.

Rustup is now natively available for the new Apple M1 devices, allowing you to install it on the new 
Macs the same way you'd install it on other platforms!

Note that at the time of writing this blog post the aarch64-apple-darwin compiler is at Tier 2 target: 
precompiled binaries are available starting from Rust 1.49 (currently in the beta channel), but no 
automated tests are executed on them.

Looks like you must compile with the Beta channel 1.49.

@frewsxcv
Copy link
Contributor

frewsxcv commented Dec 8, 2020

I opened a pull request that enables shaderc for Apple Silicon targets. All we need besides that is a new version of winit published! rust-windowing/winit#1789

@frewsxcv
Copy link
Contributor

frewsxcv commented Dec 9, 2020

#1027 has merged. Now we wait for rust-windowing/winit#1793

@frewsxcv
Copy link
Contributor

Alright, this should be the last pull request needed for Apple Silicon support: #1043

@cart
Copy link
Member

cart commented Dec 13, 2020

Great work everyone!

@enfipy
Copy link
Contributor

enfipy commented Jan 6, 2021

Recently, I've got an m1 mac and tried to build some bevy examples. Got shaderc errors while compiled bevy.

Screenshot 2021-01-06 at 10 18 55

@cart
Copy link
Member

cart commented Jan 6, 2021

Dang I thought #1027 had shaderc working on m1s

@enfipy
Copy link
Contributor

enfipy commented Jan 6, 2021

Well, I'm planning to reinstall my operating system and set up everything again tonight. Maybe this will help o_o

@frewsxcv
Copy link
Contributor

@enfipy Do you get the same error?

@enfipy
Copy link
Contributor

enfipy commented Jan 19, 2021

@frewsxcv Finally I managed to reinstall macOS and got the same error. Steps to reproduce this error on a clean M1 device:

  1. Install Rust with curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh.
  2. Clone bevy repo.
  3. Run cargo run --example ui

Screenshot 2021-01-19 at 19 27 03

The last time I tried to install cmake - I got pretty poor same results but with installed cmake for x86_64.

@frewsxcv
Copy link
Contributor

What happens if you install the ARM version of cmake? What happens if you do file $(which cmake)?

@enfipy
Copy link
Contributor

enfipy commented Jan 19, 2021

What is the right way to install cmake for ARM?

Result for file $(which cmake):

cmake: cannot open `cmake' (No such file or directory)
not:   cannot open `not' (No such file or directory)
found: cannot open `found' (No such file or directory)

@frewsxcv
Copy link
Contributor

Do you have Homebrew installed? If so, brew install cmake

@enfipy
Copy link
Contributor

enfipy commented Jan 19, 2021

Now got this error:

 [ 44%] Built target build-version

  --- stderr
  CMake Warning (dev) at CMakeLists.txt:8 (project):
    Policy CMP0048 is not set: project() command manages VERSION variables.
    Run "cmake --help-policy CMP0048" for policy details.  Use the cmake_policy
    command to set the policy and suppress this warning.

    The following variable(s) would be set to empty:

      CMAKE_PROJECT_VERSION
      CMAKE_PROJECT_VERSION_MAJOR
      CMAKE_PROJECT_VERSION_MINOR
      CMAKE_PROJECT_VERSION_PATCH
  This warning is for project developers.  Use -Wno-dev to suppress it.

  CMake Warning (dev):
    Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
    --help-policy CMP0042" for policy details.  Use the cmake_policy command to
    set the policy and suppress this warning.

    MACOSX_RPATH is not specified for the following targets:

     SPIRV-Tools-shared
     shaderc_shared

  This warning is for project developers.  Use -Wno-dev to suppress it.

  clang: error: cannot use 'precompiled-header' output with multiple -arch options
  make[2]: *** [glslang/glslang/CMakeFiles/MachineIndependent.dir/cmake_pch_x86_64.hxx.pch] Error 1
  make[1]: *** [glslang/glslang/CMakeFiles/MachineIndependent.dir/all] Error 2
  make[1]: *** Waiting for unfinished jobs....
  make: *** [all] Error 2
  thread 'main' panicked at '
  command did not execute successfully, got: exit code: 2

  build script failed, must exit now', /Users/enfipy/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.45/src/lib.rs:894:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

@darthdeus
Copy link
Contributor Author

I've definitely had a shaderc related build fail (even today), but it seems cargo clean or just doing --release or removing --release fixes it. I know this isn't probably super useful, but just wanted to confirm that it is possible to make it work on M1.

Unfortunately I can't really reproduce the failure anymore, running cmake 3.19.3 installed via homebrew. Have you tried running Rust nightly? I've only tried it on 1.51.0

@enfipy
Copy link
Contributor

enfipy commented Jan 20, 2021

@darthdeus Tested on 1.49.0 stable with and without --release flag after cargo clean - got the same error.

Tried to build with rust 1.51.0 nightly but still with that error.

Command cmake --version results in:

cmake version 3.19.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

Also, installed it with brew as specified @frewsxcv above.

@bjorn3
Copy link
Contributor

bjorn3 commented Jan 20, 2021

What does file $(which cmake) give now? That should show if cmake is built for arm. If any parent of clang is an x86_64 executable running using Rosetta, it will forcefully build x86_64 binaries.

@enfipy
Copy link
Contributor

enfipy commented Jan 20, 2021

@bjorn3 It results in:

$ file $(which cmake)
/opt/homebrew/bin/cmake: Mach-O 64-bit executable arm64

@ostwilkens
Copy link
Contributor

I'm on an M1 Macbook Air and trying to make sense of this.

I was using the 0.4.0 bevy crate in my project, no problem.
Then I got a build error from shaderc when trying to run an example in https://github.com/bonsairobo/building-blocks.
It also uses 0.4.0 bevy, so I was confused.

Trying to identify the cause, I added this to the .toml of my project:

[target.'cfg(all(target_arch = "aarch64", target_os = "macos"))'.dependencies]
shaderc = "0.7.0"

This shouldn't make a difference, as I'm depending on bevy_render, which also contains this.
However, this caused shaderc to fail to build. And now I can't get my project to build again, no matter what I do. 💩

My cmake is an universal binary:

$ file $(which cmake)
/usr/local/bin/cmake: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
/usr/local/bin/cmake (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/local/bin/cmake (for architecture arm64):	Mach-O 64-bit executable arm64

This is the error I get from shaderc-sys:

clang: error: cannot use 'precompiled-header' output with multiple -arch options
  make[2]: *** [glslang/glslang/CMakeFiles/MachineIndependent.dir/cmake_pch_x86_64.hxx.pch] Error 1
  make[1]: *** [glslang/glslang/CMakeFiles/MachineIndependent.dir/all] Error 2
  make[1]: *** Waiting for unfinished jobs....
  make: *** [all] Error 2
  thread 'main' panicked at '
  command did not execute successfully, got: exit code: 2

It could be that my project was before, for some reason, running under rosetta? I'm not sure.
All this platform jumping is give me a headache.

@ostwilkens
Copy link
Contributor

Aha. My VS Code is running under arch i386. If I start a build from VS Code, a cargo clean is required, and i can then successfully build from a non-rosetta terminal. I need to get the arm64 version.
@bjorn3 was spot on.

@enfipy What terminal are you using? Could it be running under rosetta? Try the arch command.

@enfipy
Copy link
Contributor

enfipy commented Jan 24, 2021

@ostwilkens Wow, it worked as I made cargo clean and ran it on the default macOS terminal! Thank you so much!

@FindingMain
Copy link

I'm running into the exact same error as enfipy above. I'm using the arm64 version 3.19.6 of cmake installed via homebrew and I'm using the nightly version of the rust toolchain. I'm very much new to coding and am not sure where to go from here. Was wondering if I could get some help.

@lord
Copy link

lord commented May 12, 2021

i think this also works

cargo clean
arch -arm64 cargo run

@JakkuSakura
Copy link

I'm having problem with the audio

[dependencies]
bevy = "0.5.0"
rand = "0.7.3"
error: failed to run custom build command for `coreaudio-sys v0.2.5`

Caused by:
  process didn't exit successfully: `/Users/jack/Dev/bevy_snake/target/debug/build/coreaudio-sys-96b4b3fa55dc11b8/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=COREAUDIO_SDK_PATH
  cargo:rustc-link-lib=framework=AudioUnit
  cargo:rustc-link-lib=framework=CoreAudio
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS

  --- stderr
  thread 'main' panicked at 'Unable to find libclang: "the `libclang` shared library at /usr/local/opt/llvm/lib/libclang.dylib could not be opened: dlopen(/usr/local/opt/llvm/lib/libclang.dylib, 2): no suitable image found.  Did find:\n\t/usr/local/opt/llvm/lib/libclang.dylib: mach-o, but wrong architecture\n\t/usr/local/opt/llvm/lib/libclang.dylib: stat() failed with errno=20"', /Users/jack/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.53.3/src/lib.rs:1956:31

@bjorn3
Copy link
Contributor

bjorn3 commented Oct 15, 2021

coreaudio-sys uses rust-bindgen for generating rust bindings from the C headers. Rust-bindgen requires libclang to be available. You can use brew install llvm if you use Homebrew or port install clang if you use MacPorts to install libclang. See https://rust-lang.github.io/rust-bindgen/requirements.html

@JakkuSakura
Copy link

coreaudio-sys uses rust-bindgen for generating rust bindings from the C headers. Rust-bindgen requires libclang to be available. You can use brew install llvm if you use Homebrew or port install clang if you use MacPorts to install libclang. See https://rust-lang.github.io/rust-bindgen/requirements.html

I have already installed llvm 12 via brew install llvm, but the problem remains. Currently I disabled bevy audio and it compiles.

@bjorn3
Copy link
Contributor

bjorn3 commented Oct 15, 2021

Did find:
    /usr/local/opt/llvm/lib/libclang.dylib: mach-o, but wrong architecture

It seems like rustc and libclang are compiled for different cpu architectures. Are you using an x86_64 or AArch64 rustc installation? And did brew install an x86_64 or AArch64 version of libclang? I think you can use file /usr/local/opt/llvm/lib/libclang.dylib to check which arch is used for libclang.

@mockersf
Copy link
Member

I think with M1, you need your whole build chain to be in the same architecture. If you use a terminal or an IDE to trigger your build that's not AArch64, it will try to fall back for everything on x86_64 which may fail

@JakkuSakura
Copy link

JakkuSakura commented Oct 15, 2021

I see where the problem is, but the libclang I have is for x86_64. I could not find a aarch64 version for it.
file /usr/local/opt/llvm/lib/libclang.dylib: /usr/local/opt/llvm/lib/libclang.dylib: Mach-O 64-bit dynamically linked shared library x86_64
file $(which rustc)
/Users/jack/.cargo/bin/rustc: Mach-O 64-bit executable arm64

But even I install aarch64 version of llvm, it gave me x86_64. Seems there is no aarch64 macos llvm available.
image

➜  export ARCHFLAGS='-arch arm64'
➜  brew install llvm
Warning: llvm 12.0.1 is already installed and up-to-date.
To reinstall 12.0.1, run:
  brew reinstall llvm
➜  brew reinstall llvm
==> Downloading https://ghcr.io/v2/homebrew/core/llvm/manifests/12.0.1-1
Already downloaded: /Users/jack/Library/Caches/Homebrew/downloads/d1dfbe37b08a8d9eeb5d76a5d2b4df210a1ec07a3cb97bdfbb9946de03a7c38e--llvm-12.0.1-1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/llvm/blobs/sha256:437d9a9f70c71c2cbab49dafc2fb
Already downloaded: /Users/jack/Library/Caches/Homebrew/downloads/fa7df652c97d5dd4d9cdcc736c38b5cae4ff22873701c7f7f148b235cded0d76--llvm--12.0.1.arm64_big_sur.bottle.1.tar.gz
==> Reinstalling llvm
==> Pouring llvm--12.0.1.arm64_big_sur.bottle.1.tar.gz
==> Caveats
To use the bundled libc++ please add the following LDFLAGS:
  LDFLAGS="-L/opt/homebrew/opt/llvm/lib -Wl,-rpath,/opt/homebrew/opt/llvm/lib"

llvm is keg-only, which means it was not symlinked into /opt/homebrew,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have llvm first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc

For compilers to find llvm you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"

==> Summary
🍺  /opt/homebrew/Cellar/llvm/12.0.1: 9,803 files, 1.5GB
➜ file /usr/local/opt/llvm/lib/libclang.dylib
/usr/local/opt/llvm/lib/libclang.dylib: Mach-O 64-bit dynamically linked shared library x86_64

So it's quite the opposite. My rust toolchain is native aarch64, but llvm does not have it for macos.

@bjorn3
Copy link
Contributor

bjorn3 commented Oct 15, 2021

Try brew install --build-from-source llvm to build llvm from source. This will hopefully build it for AArch64.

@JakkuSakura
Copy link

Building, not sure how long it will take. Hopefully it will solve the problem

@JakkuSakura
Copy link

It seems to find the system wide clang as default. How can I change it to the one I previously compiled?

➜  ~ locate libclang.dylib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libclang.dylib
/Applications/Xcode.app/Contents/Frameworks/libclang.dylib
/Library/Developer/CommandLineTools/usr/lib/libclang.dylib
/usr/local/Cellar/llvm/11.0.0_1/Toolchains/LLVM11.0.0.xctoolchain/usr/lib/libclang.dylib
/usr/local/Cellar/llvm/11.0.0_1/lib/libclang.dylib
➜  ~ file /Library/Developer/CommandLineTools/usr/lib/libclang.dylib
/Library/Developer/CommandLineTools/usr/lib/libclang.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64]
/Library/Developer/CommandLineTools/usr/lib/libclang.dylib (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64
/Library/Developer/CommandLineTools/usr/lib/libclang.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64
➜  ~ export LD_LIBRARY_PATH=/Library/Developer/CommandLineTools/usr/lib:$LD_LIBRARY_PATH
➜  ~ cd Dev/the_snakes
➜  the_snakes git:(master) ✗ cargo build
   Compiling spirv_cross v0.23.1
   Compiling shaderc-sys v0.7.3
   Compiling coreaudio-sys v0.2.8
   Compiling bevy_window v0.5.0
   Compiling bevy_transform v0.5.0
   Compiling bevy_input v0.5.0
   Compiling bevy_dynamic_plugin v0.5.0
   Compiling bevy_diagnostic v0.5.0
error: failed to run custom build command for `coreaudio-sys v0.2.8`

Caused by:
  process didn't exit successfully: `/Users/jack/Dev/the_snakes/target/debug/build/coreaudio-sys-a8c3de66f2fdce6e/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=COREAUDIO_SDK_PATH
  cargo:rustc-link-lib=framework=AudioToolbox
  cargo:rustc-link-lib=framework=AudioToolbox
  cargo:rustc-link-lib=framework=CoreAudio
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS

  --- stderr
  thread 'main' panicked at 'Unable to find libclang: "the `libclang` shared library at /usr/local/Cellar/llvm/11.0.0_1/lib/libclang.dylib could not be opened: dlopen(/usr/local/Cellar/llvm/11.0.0_1/lib/libclang.dylib, 5): no suitable image found.  Did find:\n\t/usr/local/Cellar/llvm/11.0.0_1/lib/libclang.dylib: mach-o, but wrong architecture\n\t/usr/local/Cellar/llvm/11.0.0_1/lib/libclang.dylib: stat() failed with errno=20"', /Users/jack/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.56.0/src/lib.rs:1922:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-Bug An unexpected or incorrect behavior O-MacOS Specific to the MacOS (Apple) desktop operating system
Projects
None yet
Development

Successfully merging a pull request may close this issue.