Unable to find crate `proc_macro` on musl target

[golddranks]

Using procedural derive macros on musl seems to work just fine for the most of the cases, but there is one pitfall; in some cases like this dtolnay/proc-macro-hack#6 build fails because not only the crate that implements the macro (crate marked with proc-macro = true), but also another, normal crate links to the proc_macro crate. "Normal" crates are unable to find proc_macro when built on musl target:

Compiling proc-macro-hack v0.3.0
error[E0463]: can't find crate for `proc_macro`
 --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro-hack-0.3.0/src/lib.rs:1:1
  |
1 | extern crate proc_macro;
  | ^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to previous error

error: Could not compile `proc-macro-hack`.

[alexcrichton]

Yes this is currently intentional. The proc_macro crate is only a dynamic library which the musl target currently doesn't support. I unfortunately don't think this will be easy to fix...

[golddranks]

I understand why the proc macro crates need to be dynamic: they are loaded and run dynamically by the compiler. However what I don't understand is why the proc_macro crate isn't allowed to be linked both dynamically and statically? I'd imagine someone may want to use the types defined in proc_macro in some helper crate, outside the dynamically linked macro crate, as is the case here.

[golddranks]

Is there some technical hurdle that makes it hard to do, or is it just not done because it's such a minor case?

[golddranks]

The simplest test for this seems to be:

test.rs with just single line:

extern crate proc_macro;

build with (succeeds):

rustc test.rs --crate-type lib

and with (fails):

rustc test.rs --crate-type lib --target=x86_64-unknown-linux-musl

When RUST_LOG=info is enabled, the library probes can be seen.

[golddranks]

Okay, I did some testing.

I git cloned the Rust source, edited away the crate-type = ["dylib"] annotation in libproc_macro, librustc_data_structures, librustc_errors, libsyntax and libsyntax_pos. (Transitive deps of libproc_macro)

After that, I built a rlib of libproc_macro with cargo build --target=x86_64-unknown-linux-musl. (This is on the newest nightly.)

After that, I copied the build artefacts from ~/rust/src/target/x86_64-unknown-linux-musl/debug/deps/ to ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib.

After that, I try to compile the things that failed before with musl target → it works.

So, it seems that there isn't any real technical hurdles. (or if there is, I haven't encountered them yet)

@alexcrichton, what do you think of distributing rlibs of aforementioned libraries with the musl target?

[golddranks]

Btw. I did this work on a Linux host. Tried to do the same on macOS (with musl target of course), but on macOS the linking fails with ld: unknown option: --as-needed. The linker is passed a flag --as-needed which seems to be supported on GNU linker only.

[golddranks]

Ah, okay, the --as-needed behaviour is explained here: #34282

[alexcrichton]

Yes there are technical hurdles in terms of build system of doing that. This is also rarely what you want because there's no reason to have a full Rust parser linked into a binary (when it's not used).

[golddranks]

I see, that's a convincing argument.

[drozdziak1]

Hello,
sorry to dig up a closed issue, but I'm experiencing the same problem during an OpenWrt cross build (i686-unknown-linux-musl is the Rust target and VirtualBox is the OpenWrt one). My compiler is rustc 1.25.0-nightly (61452e506 2018-01-09). I've got a couple questions regarding this:

• Is the root cause known? What is it?
• Is there a solution out there?
• If not, where can I go to help solve this?

[lisbakke]

Hello! I've got this issue, as well -- deps relying on proc-macro2 which do not use default-features = false so proc-macro crate is not found because my target does not have it.

I'm currently compiling rustc for my target and it seems easiest to create proc-macro lib when I'm building. Is there any documentation on how to do this?

[mmastrac]

For those following various links to this issue, it is important to note that you can compile code using proc_macro to statically-linked musl binaries, BUT you need to cross-compile from a platform that does support dynamic linking.

cargo build --target x86_64-unknown-linux-musl should be sufficient, but you'll need to run that on something like Ubuntu, Fedora, or another full system.

[jonas-schievink]

Reopening as this seems to still be a significant issue for people building from musl systems