allow dynamic lookup of symbols on macOS?

[kevinushey]

On macOS, one can use the linker flags -undefined dynamic_lookup to basically tell macOS that the application will provide the required symbols at runtime, rather than via explicitly linking with some library at build time.

This is mainly relevant for applications which embed R, but want to be able to "load" different versions of R at runtime. Otherwise, the library location will be embedded in the compiled library / application, and one cannot (easily) switch to a different version.

If I understand correctly, libR-sys declares that it will link to libR.dylib here:

libR-sys/build.rs

Lines 491 to 493 in d91909b

	// make sure cargo links properly against library
	println!("cargo:rustc-link-search={}", r_paths.library.display());
	println!("cargo:rustc-link-lib=dylib=R");

Would you consider a PR that relaxes this requirement (perhaps behind a separate feature)?

[Ilia-Kosenkov]

Hey! If you have a solution at hand to your problem, let's see it. extendr has an extensive set of tests targeting all platforms, if your changes are compatible with it, I see no problem in including it, especially feature-guarded.
Other members of the core team might voice their concerns though, but we are open-minded :)

[kevinushey]

Here's one solution I've been testing: kevinushey@3f022f2

I still need to do some more reading to better understand if this is the right way to approach the problem, since there's a number of questions I don't know the answer to yet:

• Should this be hidden behind a feature e.g. dynamic-lookup, which presumedly would not be enabled by default?
• If so, are those the right linker arguments? (Since presumedly, this would need to be passed along to any dependency linking to libR-sys, including recursive dependencies, I think?)
• Cargo lets you override the build scripts for certain dependencies as described in https://doc.rust-lang.org/cargo/reference/build-scripts.html#overriding-build-scripts. Is there a way forward using that mechanism? If so, that might be preferred since it wouldn't require any changes in libR-sys; however, I'm not sure if there's a way to make this work with the way bindgen bindings are created / copied in the libR-sys build script.

[Ilia-Kosenkov]

Tricky questions, I, unfortunately, do not have the capacity to answer everything.

• Feature-guard is important since I assume this changes a lot in how linking is done on MacOs. I'd add macos or something like that in the name of the feature to indicate it is os-specific
• No idea if they are correct. However, if it compiles successfully, I guess it is a good sign. One way would be to drop these arguments completely, observe compilation/execution issues, then add your changes and hopefully observe successful compilation & execution. This ensures that you indeed switched to another linking process.
• I am aware of that but we use it only in a select few places. Right now I doubt we can use that (which means that we have no tools to support that right now/did not test it/did not consider it before).

The R-Rust interop is an exciting journey, but for us, it is done mostly through trial&error. If you change it & it works (and more importantly, it works as you expect), it is good enough to start the PR process.

[kevinushey]

Thanks! I appreciate the quick and thoughtful response.

I'll try to investigate a bit more and see if I can put together a PR.

[kevinushey]

I spent some more time looking at this, and I think it ultimately becomes a bit too complicated to thread these linker flags through through both libR-sys and the dependencies of libR-sys. (It also becomes a headache once you think about running cargo test, or considering how these flags should propagate in other build configurations.)

Instead, I think a workable solution for users who need this is to just post-process the generated executable; e.g. using install_name_tool to change the path to libR.dylib to some other location (even a non-existent location appears to be fine if one ensures that DYLD_INSERT_LIBRARIES is set appropriately before launching the executable embedding R).

[kevinushey]

And just to close the thread (for anyone else who stumbled here...) other applications using Rust to embed other libraries recommend using .cargo/config.toml as follows:

[target.x86_64-apple-darwin]
rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
]

[target.aarch64-apple-darwin]
rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
]

It's not perfect (since these flags apply to all link invocations) but in practice it's fine.

[yutannihilation]

I thInk it can be set via CARGO_BUILD_RUSTFLAGS envvar. I use a CARGO_ envvar instead if .cargo/config.toml:

https://github.com/extendr/libR-sys#tweak-the-toolchain