Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MacOS fat binaries support #416

Open
xTachyon opened this issue Jul 7, 2023 · 8 comments
Open

MacOS fat binaries support #416

xTachyon opened this issue Jul 7, 2023 · 8 comments

Comments

@xTachyon
Copy link

xTachyon commented Jul 7, 2023

I'm trying to compile a project under MacOS that sets -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64" in order to create a fat binary. The code compiles, but in the linking step, it fails with the below message, plus a bunch of undefined symbols:

ld: warning: ignoring file /Users/.../therustlib.a, building for macOS-arm64 but attempting to link with file built for macOS-x86_64

I've tried messing with Rust_CARGO_TARGET, but every time the build fails on a mismatch or another. I've even tried putting both the targets in that variable, but it doesn't seem like it's something that the corrosion accepts.

I've tried detecting the arch that we're currently compiling to with stuff like CMAKE_SYSTEM_PROCESSOR, but this seems to only be ever set to x86_64.

@jschwe
Copy link
Collaborator

jschwe commented Jul 7, 2023

As far as I know cargo/rust doesn't support compiling fat binaries (unless this changed recently)

@xTachyon
Copy link
Author

xTachyon commented Jul 7, 2023

A possible solution I can see is compiling both versions separately, and then merging them with lipo. I think that's what the xcode generator is doing.

I've tried doing that manually for a test, and hardcoding the .a path in cmake, and it seems to have worked. Maybe corrosion can automate this?

@jschwe
Copy link
Collaborator

jschwe commented Jul 8, 2023

s compiling both versions separately, and then merging them with lipo.

What is the scope here? Does this work for all rust artifacts corrosion supports, i.e. for staticlibs AND for cdylibs AND for bin targets? Your example seems to imply you only tested with staticlibs.

In principle it would be possible for corrosion to support this, but i'd have to give some more thought on how much refactoring is required to allow this. Currently, the assumption of "one target triple" is pretty baked in.

If you just want a quick solution for your own project, then you could probably just configure two new sub-cmake projects from your main cmake project, and add a rule in the main cmake project to use lipo to merge the artifacts from the sub cmake project.

@xTachyon
Copy link
Author

xTachyon commented Jul 8, 2023

Initially I just tested with staticlibs, but just now I tested with a new hello world project with bin, staticlib and cdylib by building them twice and invoking lipo on them and it seems to be working on all the artifact types. sh for cdylib:

cargo b --target x86_64-apple-darwin &&
cargo b --target aarch64-apple-darwin &&
lipo -create target/x86_64-apple-darwin/debug/libhello_world.dylib target/aarch64-apple-darwin/debug/libhello_world.dylib -output libhello_world_fat.dylib

file output:

% file libhello_world_fat.dylib
libhello_world_fat.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64:Mach-O 64-bit dynamically linked shared library arm64]
libhello_world_fat.dylib (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64
libhello_world_fat.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64

The syntax for lipo seems to be identical in all 3 cases, I just changed the input and output filenames and it seems to pick up what they are automagically.
So yes, it could work with all already supported crate types.

About your quick solution, could you please elaborate more please? What I understood right now was that I should do 2x corrosion_import_crate, and then do the lipo step myself, which sounded like a good workaround, but importing the same crate twice doesn't seem to be supported because "a target with the same name already exists".

@xTachyon xTachyon changed the title Compiling under MacOS with multi arch build picks the wrong arch .a MacOS fat binaries support Jul 8, 2023
@jschwe
Copy link
Collaborator

jschwe commented Jul 11, 2023

Thanks for testing!

What I understood right now was that I should do 2x corrosion_import_crate

No that doesn't work, because corrosion currently can only import each target once.

What I had in mind was something like

add_custom_command(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild/mylib.a"
     COMMAND ${CMAKE_COMMAND} -S "${CMAKE_CURRENT_SOURCE_DIR}/rust"
                 -B "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild"
                 -DRust_CARGO_TARGET=....
     COMMAND ${CMAKE_COMMAND} --build "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild"
)

This custom command would be added once for each target and assumes that "${CMAKE_CURRENT_SOURCE_DIR}/rust" contains a CMakeLists.txt (with its own project() ) which adds corrosion and calls corrosion_import_crate().

In your main CMake files you could then add another custom command which DEPENDs on the output files of the commands, and calls lipo on them.

@xTachyon
Copy link
Author

xTachyon commented Aug 8, 2023

Found out you can also pass --target multiple times to cargo, and it will know how to build multiple arches at a time. In parallel even!

@jschwe
Copy link
Collaborator

jschwe commented Aug 9, 2023

Nice find - do you happen to know starting with which Rust version this works?

Edit: Seems to be Rust 1.64

@xTachyon
Copy link
Author

xTachyon commented Aug 9, 2023

Looks like 1.64.

Edit: you found first while I was searching😋.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants