macOS: Apple linker not aware of the default 10.7 deployment target, causing issues on macOS 12.0 Monterey

[cormacrelf]

TL;DR for passers-by who just want to compile code on Monterey

Try this, it might fix an error like ld: reference to symbol (which has not been assigned an address) or ld: Assertion failed: (_mode == modeFinalAddress), function finalAddress, file ld.hpp, line 1190.

export MACOSX_DEPLOYMENT_TARGET=10.7
cargo clean
cargo build/run/etc

Core problem

macOS targets are tricky because LLVM behaves differently depending on the MACOSX_DEPLOYMENT_TARGET environment variable, or a version specified in the target triple you tell LLVM to use.

However, another component also uses the deployment target information to customise its output. That is the linker, ld from Xcode / the Command Line Tools.

Rustc's default deployment target is 10.7. It only passes this to LLVM, and not to ld.

When you invoke rustc using env MACOSX_DEPLOYMENT_TARGET=10.7 cargo/rustc/etc, it does what it should be doing by default, because it allows that env var to pass through to ld (aka cc). When you do not provide the environment variable, it results in LLVM using 10.7 but ld using a much, much newer one.¹ I believe this to be a bug in its own right -- you would expect rustc's default deployment target to apply to both the compiler and the Apple linker, but it does not.

Solution: set MACOSX_DEPLOYMENT_TARGET=10.7 if it is not already present in the environment for the cc invocation that ultimately calls ld to create a finished binary.

Observing this in practice / a linker error repro

I found this when compiling https://lib.rs/curl on an M1/aarch64 Monterey machine. It involves the link_section="__DATA,__mod_init_func" technique see e.g. here. The cause of the error is that with the 'much, much newer' deployment target that ld uses by default, the linker transforms this into something completely different, and it prevents linking to the static function pointer, something that works with any deployment target set.

But without further ado, this fails with a linker error if you build it in debug mode on macOS 12.0, using Xcode 13 or the Command Line Tools. It is very similar to this code in curl-rust.

// lib.rs
#[cfg_attr(target_os = "macos", link_section = "__DATA,__mod_init_func")]
#[used]
static INIT_FUNC: extern "C" fn() = init_func;

extern "C" fn init_func() {
    println!("And if the band you're in starts playing different tunes");
    println!("I'll see you on the dark side of the main.\n");
}

pub fn init_manually() {
    INIT_FUNC();
}

// main.rs
fn main() {
    println!("Hello from main()\n");
    // this should simply print the message a second time.
    // including this line works around <https://github.com/rust-lang/rust/issues/47384> but it
    // causes a linker error, when compiling the call to INIT_FUNC
    mod_init_func::init_manually();
}

cargo build # fails with ld error, below
cargo build --release # works fine, because LLVM has managed to inline `init_func` into `init_manually`.

The error message is a bit fragile, it seems to depend on whether you are compiling a finished binary or some intermediate crate. When compiling a crate that depends on curl, you get this:

$ cargo build

... long cc -arch arm64 invocation
...
  = note: ld: reference to symbol (which has not been assigned an address) __ZN4curl4init9INIT_CTOR17h97cc33cf050cb462E in '__ZN4curl4init17ha644d831c2a57f65E' from /Users/cormac/git/tryout/libcurl-monterey/target/debug/deps/libcurl-0f9cbb7dde66dd88.rlib(curl-0f9cbb7dde66dd88.curl.0b6dcf6e-cgu.2.rcgu.o) for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

But with this repro or cargo test in the curl-rust repo, you get this (very similar in spirit):

  = note: 0  0x100340224  __assert_rtn + 128
          1  0x1003457e8  ld::tool::OutputFile::addressAndTarget(ld::Internal const&, ld::Fixup const*, ld::Atom const**) (.cold.1) + 0
          2  0x10027f104  ld::tool::OutputFile::addressOf(ld::Internal const&, ld::Fixup const*, ld::Atom const**) + 252
          3  0x100280478  ld::tool::OutputFile::applyFixUps(ld::Internal&, unsigned long long, ld::Atom const*, unsigned char*) + 1568
          4  0x100285540  ld::tool::OutputFile::writeAtoms(ld::Internal&, unsigned char*) + 356
          5  0x10027cfa4  ld::tool::OutputFile::writeOutputFile(ld::Internal&) + 408
          6  0x100275adc  ld::tool::OutputFile::write(ld::Internal&) + 216
          7  0x1002031d8  main + 584
          A linker snapshot was created at:
                /tmp/mod_init_func-0236ccc15f993087-2021-09-27-224626.ld-snapshot
          ld: Assertion failed: (_mode == modeFinalAddress), function finalAddress, file ld.hpp, line 1190.
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

Bonus: Linkers and pre-main init hooks in Mach-O

There appear to be some changes around this recently. If you compile the equivalent C code, you actually get the exact same problem.

Using ld from Apple LLVM 13.0.0 on a Monterey machine, linking a C file with __attribute__((section("__DATA,__mod_init_func"))) typeof(myinit) *__init = myinit;

MACOSX_DEPLOYMENT_TARGET	where it ends up	runs before main	you can call __init(...) as a function from main
10.7 through 10.14	__DATA,__mod_init_func	✅	✅
10.15 through 12.0	__DATA_CONST,__mod_init_func	✅	✅
not set	__TEXT,__init_offsets, with a completely different format	✅	❌ - "ld: reference to symbol (which has not been assigned an address) ___init in '_main'"

If you tell clang to link the static in the __DATA_CONST,__mod_init_func section instead, then it doesn't work at all, it doesn't run before main. Clearly the "API" is to use the well-known __DATA,__mod_init_func, and the only guarantee is that it will execute that function before main.

The above particular Rust code not compiling is therefore not really a rustc bug in its own right. Every platform has its own way of doing this, and "newer macOS" is just another variation that needs to be added. Hacky platform-specific linker section stuff is almost certainly out of scope for stable/guaranteed behaviour. To do this correctly I think you would need a build.rs that always has access to a MACOSX_DEPLOYMENT_TARGET env variable, i.e. cargo should set the env var to 10.7 if it is not already set. Then you could set some cfgs in build.rs to determine which link section to add when the target_os is macos. That solution also works for informing ld, the only difference being build.rs might get it from cargo.

Meta

rustc --version --verbose:

rustc 1.56.0 (09c42c458 2021-10-18)
binary: rustc
commit-hash: 09c42c45858d5f3aedfa670698275303a3d19afa
commit-date: 2021-10-18
host: aarch64-apple-darwin
release: 1.56.0
LLVM version: 13.0.0

This happens in beta/nightly-2021-10-26 too.

ld version:

ld -v
@(#)PROGRAM:ld  PROJECT:ld64-711
BUILD 18:11:19 Aug  3 2021
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 13.0.0, (clang-1300.0.29.3) (static support for 27, runtime is 27)
TAPI support using: Apple TAPI version 13.0.0 (tapi-1300.0.6.5)

Footnotes

1. it's not 12.0! It's different, somehow even newer! See the table. ↩

[cormacrelf]

@rustbot label O-macos A-linkage

[hkratz]

I ran into this, but only with the ld64 that comes with Xcode 13.1 not with Xcode 13. Are you sure you can reproduce it with Xcode 13 as well?

[cormacrelf]

Oh, indeed, I am running 13.1. I don't know about Xcode 13.

[bdgrichards]

Thanks so much for this! Took a few hours of debugging before I finally ran into this page, solved it for me

[hkratz]

@rustbot claim

[syrusakbary]

Thanks for reporting this. I had an issue when compiling wasm-bindgen in macOS Monterey

$ cargo install wasm-pack
# ...
          ld: Assertion failed: (_mode == modeFinalAddress), function finalAddress, file ld.hpp, line 1190.
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

But following the suggestion on the issue fixed it!

$ export MACOSX_DEPLOYMENT_TARGET=10.7
$ cargo install wasm-pack
# ...
   Installed package `wasm-pack v0.10.1` (executable `wasm-pack`)

[sagebind]

We've worked around this issue in the curl crate so that we won't trigger this bug -- curl 0.4.41 should now compile on Monterey without needing to use the environment variable workaround. Though this bug still ought to be fixed.

[russweas]

I'm no longer having this issue, even without updating curl to 0.4.41.

[cormacrelf]

@russweas Are you sure that's not just the absence of a lockfile, with cargo just fetching latest curl from crates.io?