Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Apple Silicon (M1/M2) support #128

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

FtZPetruska
Copy link

@FtZPetruska FtZPetruska commented Jun 19, 2023

Compiling natively on my M1 machine failed with the following issues:

  • CONFIGURE_COMMAND using ${compiler_flags} ${command_wrapper} fail due to _CFLAGS being treated as a command rather than an environment variable.
  • Dependencies built with autotools not knowing about the arm64/aarch64-apple-darwin triple.
  • Project-specific issues.

Moreover, the latest macOS x64 release no longer works on Apple Silicon due to missing shared libraries.

For the CONFIGURE_COMMAND issue, the fix consists in invoking the command_wrapper using cmake -E env. This achieves the expected behaviour where the compiler_flags variables are treated as environment variables, and passed to the command_wrapper script.

A similar issue happened for vita-headers with the call to python, starting macOS 12.3 there is no longer a python binary in PATH. To fix this, I used CMake to find the Python executable and used it instead.

For the dependencies, they typically needed to be updated to their latest version so the platform could be correctly detected.

Two notable exceptions were gmp and libelf:

  • For gmp, the patch used by Homebrew is used. Since GMP 6.3.0, running autoreconf has the same effect as the patch.
  • For libelf, the config.guess and config.sub files are updated using the ones from automake 1.16.

Lastly, there were some project-specific issues:

  • zlib shared libraries would not be deleted
  • I had difficulties downloading libyaml from the pyyaml website, so I switched the download source to github. It is functionally identical.
  • gdb did not declare its dependencies on the other projects, and would also fail to find said dependencies.

With these changes, I was able to build the toolchain natively on my M1 mac, compile the SDL2 port and link it against executables.

A few additional notes:

  • Use the correct triple for arm64 #126 is not strictly required for this PR, most config.sub scripts are able to canonicalise arm64-apple-darwin into aarch64-apple-darwin.
  • mpfr provides a cumulative patch for version 4.2.0 that addresses a handful of bugs. This patch is included in this PR. The patch is no longer needed for 4.2.1.
  • CI should be able to build an Apple Silicon version, but it would require additional changes (namely, making sure to pass the correct -target CFLAGS to every configure script). Let me know if this should be part of this PR or a subsequent one.

This PR closes #101, closes #107, and closes #122.

@FtZPetruska
Copy link
Author

FtZPetruska commented Jul 29, 2023

Alright, rebased on master to get the latest changes, fixed the conflicts and updated this PR a bit.

To sum up the changes:

  • binutils and gcc were (re?)updated to 2.40 2.41 and 13.1.0 13.2.0 respectively, the current versions did not support Apple Silicon.
  • Build zstd. Recent versions of GCC, Binutils, and GDB received support for zstd-compressed debug section. Building it ourselves avoids picking up a system installation (most notably in CI), and also users can enjoy up-to-date feature! It does require CMake 3.7, due to the necessary use of SOURCE_SUBDIR in ExternalProject.
  • Update the mpfr cumulative patch from p9 to p12.

Here's a sample CI run if you wish to test the binaries.

@FtZPetruska FtZPetruska marked this pull request as ready for review July 29, 2023 03:12
This allows detecting newer platforms like Apple Silicon.

Also update the commands to use `cmake -E env`.
- Download from github rather than pyyaml.
- Update to the latest version (0.2.5).
- Use `cmake -E env` to invoke the wrapper command.
- Run autoreconf to fix flat namespace issues on macOS 11+.
- Don't use the command wrapper to build, this breaks on arm64 linux.
@zetanumbers
Copy link
Contributor

Using vita-toolchain built with this PR (after rebasing it onto master). Haven't had any issues yet.

This adds support for Apple Silicon.
This adds Apple Silicon support.
This is needed by the latest gcc, binutils, and gdb, and will avoid
using the host's version.
- The version update provides support for Apple Silicon.
- The patch offsets were also updated.
- The configure command had to be updated to find GMP/MPFR/MPC. The GDB
configure script called during build uses different flags to set lib
paths.
This adds support for Apple Silicon and zstd.
- This adds support for Apple Silicon hosts.
- The patch was renamed (and updated) to reflect the change.
- Add zstd support.
- zstd external project requires CMake 3.7, so GIT_SHALLOW can be used
unconditionally.
- Use `option` to declare OFFLINE.
- Expand `OFFLINE` in the ExternalProject calls instead of using an
intermediate variable.
- Set CMP0135 to NEW.
Binutils 2.41 requires a version newer than what is provided by macOS.
Also run autoreconf to fix a sporadic issue with automake.
Use the github URL as it will not change when the next version releases.
@FtZPetruska
Copy link
Author

FtZPetruska commented Sep 18, 2023

Thank you for your feedback!

I just rebased on master. While doing so, I noticed several more issues when running CI on my fork, namely:

  • Using the command wrapper to build GMP would break during linking due to a race condition. Its use should not be necessary since the flags were already passed during configure.
  • expat would sporadically break in the Windows and Linux builds due to an automake version mismatch. Running autoreconf fixes the issue.
  • mpfr had the same issue, running autoreconf also handled it.

For zlib, I also switched the URL to the github release, so it does not break when the next release comes out.


Here's the latest CI run on my fork if you want to test the builds.

@zetanumbers
Copy link
Contributor

zetanumbers commented Sep 28, 2023

Here's the latest CI run on my fork if you want to test the builds.

It says it's Mach-O 64-bit executable x86_64

UPDATE: i have built vitasdk from deps-upgrade. Using it for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants