Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to find crate `proc_macro` on musl target #40174

Open
golddranks opened this issue Mar 1, 2017 · 13 comments
Open

Unable to find crate `proc_macro` on musl target #40174

golddranks opened this issue Mar 1, 2017 · 13 comments

Comments

@golddranks
Copy link
Contributor

@golddranks golddranks commented Mar 1, 2017

Using procedural derive macros on musl seems to work just fine for the most of the cases, but there is one pitfall; in some cases like this dtolnay/proc-macro-hack#6 build fails because not only the crate that implements the macro (crate marked with proc-macro = true), but also another, normal crate links to the proc_macro crate. "Normal" crates are unable to find proc_macro when built on musl target:

Compiling proc-macro-hack v0.3.0
error[E0463]: can't find crate for `proc_macro`
 --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro-hack-0.3.0/src/lib.rs:1:1
  |
1 | extern crate proc_macro;
  | ^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to previous error

error: Could not compile `proc-macro-hack`.
@alexcrichton

This comment has been minimized.

Copy link
Member

@alexcrichton alexcrichton commented Mar 1, 2017

Yes this is currently intentional. The proc_macro crate is only a dynamic library which the musl target currently doesn't support. I unfortunately don't think this will be easy to fix...

@golddranks

This comment has been minimized.

Copy link
Contributor Author

@golddranks golddranks commented Mar 1, 2017

I understand why the proc macro crates need to be dynamic: they are loaded and run dynamically by the compiler. However what I don't understand is why the proc_macro crate isn't allowed to be linked both dynamically and statically? I'd imagine someone may want to use the types defined in proc_macro in some helper crate, outside the dynamically linked macro crate, as is the case here.

@golddranks

This comment has been minimized.

Copy link
Contributor Author

@golddranks golddranks commented Mar 1, 2017

Is there some technical hurdle that makes it hard to do, or is it just not done because it's such a minor case?

@golddranks

This comment has been minimized.

Copy link
Contributor Author

@golddranks golddranks commented Mar 1, 2017

The simplest test for this seems to be:

test.rs with just single line:

extern crate proc_macro;

build with (succeeds):

rustc test.rs --crate-type lib

and with (fails):

rustc test.rs --crate-type lib --target=x86_64-unknown-linux-musl

When RUST_LOG=info is enabled, the library probes can be seen.

@golddranks

This comment has been minimized.

Copy link
Contributor Author

@golddranks golddranks commented Mar 1, 2017

Okay, I did some testing.

I git cloned the Rust source, edited away the crate-type = ["dylib"] annotation in libproc_macro, librustc_data_structures, librustc_errors, libsyntax and libsyntax_pos. (Transitive deps of libproc_macro)

After that, I built a rlib of libproc_macro with cargo build --target=x86_64-unknown-linux-musl. (This is on the newest nightly.)

After that, I copied the build artefacts from ~/rust/src/target/x86_64-unknown-linux-musl/debug/deps/ to ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib.

After that, I try to compile the things that failed before with musl target → it works.

So, it seems that there isn't any real technical hurdles. (or if there is, I haven't encountered them yet)

@alexcrichton, what do you think of distributing rlibs of aforementioned libraries with the musl target?

@golddranks

This comment has been minimized.

Copy link
Contributor Author

@golddranks golddranks commented Mar 1, 2017

Btw. I did this work on a Linux host. Tried to do the same on macOS (with musl target of course), but on macOS the linking fails with ld: unknown option: --as-needed. The linker is passed a flag --as-needed which seems to be supported on GNU linker only.

@golddranks

This comment has been minimized.

Copy link
Contributor Author

@golddranks golddranks commented Mar 1, 2017

Ah, okay, the --as-needed behaviour is explained here: #34282

@alexcrichton

This comment has been minimized.

Copy link
Member

@alexcrichton alexcrichton commented Mar 2, 2017

Yes there are technical hurdles in terms of build system of doing that. This is also rarely what you want because there's no reason to have a full Rust parser linked into a binary (when it's not used).

@golddranks

This comment has been minimized.

Copy link
Contributor Author

@golddranks golddranks commented Mar 2, 2017

I see, that's a convincing argument.

@golddranks golddranks closed this Mar 2, 2017
@drozdziak1

This comment has been minimized.

Copy link

@drozdziak1 drozdziak1 commented Jan 10, 2018

Hello,
sorry to dig up a closed issue, but I'm experiencing the same problem during an OpenWrt cross build (i686-unknown-linux-musl is the Rust target and VirtualBox is the OpenWrt one). My compiler is rustc 1.25.0-nightly (61452e506 2018-01-09). I've got a couple questions regarding this:

  • Is the root cause known? What is it?
  • Is there a solution out there?
  • If not, where can I go to help solve this?
malbarbo added a commit to malbarbo/tectonic that referenced this issue Nov 15, 2018
Musl target does not work with proc-macro.
See rust-lang/rust#40174

Removing serde-derive allows building tectonic for musl targets.
@lisbakke

This comment has been minimized.

Copy link

@lisbakke lisbakke commented Nov 21, 2018

Hello! I've got this issue, as well -- deps relying on proc-macro2 which do not use default-features = false so proc-macro crate is not found because my target does not have it.

I'm currently compiling rustc for my target and it seems easiest to create proc-macro lib when I'm building. Is there any documentation on how to do this?

@mmastrac

This comment has been minimized.

Copy link

@mmastrac mmastrac commented Oct 6, 2019

For those following various links to this issue, it is important to note that you can compile code using proc_macro to statically-linked musl binaries, BUT you need to cross-compile from a platform that does support dynamic linking.

cargo build --target x86_64-unknown-linux-musl should be sufficient, but you'll need to run that on something like Ubuntu, Fedora, or another full system.

@jonas-schievink

This comment has been minimized.

Copy link
Member

@jonas-schievink jonas-schievink commented Nov 13, 2019

Reopening as this seems to still be a significant issue for people building from musl systems

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.