support proc-macros

[dvc94ch]

#[unroll::unroll_for_loops]
fn unroll_loop(a: Vec3) -> Vec3 {
    let mut b = Vec3::default();
    for i in 0..3 {
        b[i] + 1.0;
    }
    b
}

cargo +nightly build                                       (base) 
   Compiling wgpu-example-runner v0.1.0 (/home/dvc/implicit/marching_rays/runner)
error: failed to run custom build command for `wgpu-example-runner v0.1.0 (/home/dvc/implicit/marching_rays/runner)`

Caused by:
  process didn't exit successfully: `/home/dvc/implicit/marching_rays/target/debug/build/wgpu-example-runner-fbbe94bc606d1aa7/build-script-build` (exit code: 1)
  --- stderr
     Compiling wgpu-example-shader v0.1.0 (/home/dvc/implicit/marching_rays/shader)
  error[E0463]: can't find crate for `unroll`
    --> shader/src/lib.rs:45:3
     |
  45 | #[unroll::unroll_for_loops]
     |   ^^^^^^ can't find crate
     |
     = note: extern location for unroll is of an unknown type: /home/dvc/implicit/marching_rays/target/release/deps/libunroll-75b6e6a2c714c829.so
     = help: file name should be lib*.rlib or *..spv

  error: aborting due to previous error

  For more information about this error, try `rustc --explain E0463`.
  error: could not compile `wgpu-example-shader`

  To learn more, run the command again with --verbose.
  Error: BuildFailed

[nolanderc]

I'm going to look into this!

[nolanderc]

I traced the issue down to some weird behavior in rustc. The gist of the issue is that when building with the spir-v backend we configure the compiler to generate .spv files:

rust-gpu/crates/rustc_codegen_spirv/src/lib.rs

Lines 163 to 166 in afe3e1f

	options: TargetOptions {
	dll_prefix: "".to_string(),
	dll_suffix: ".spv".to_string(),
	panic_strategy: PanicStrategy::Abort,

However, for some reason, rustc uses the same postfix and suffix (*.spv) when considering which libraries it can use for proc-macros, instead of using the default for the system it's running on (*.so on Linux, like in the error above). I tried changing the lines dll_suffix and dll_prefix above to match my system (MacOS) but got a similar error:

     Compiling core v0.0.0 (/Users/christofer/.rustup/toolchains/nightly-2020-10-25-x86_64-apple-darwin/lib/rustlib/src/rust/library/core)
     Compiling rustc-std-workspace-core v1.99.0 (/Users/christofer/.rustup/toolchains/nightly-2020-10-25-x86_64-apple-darwin/lib/rustlib/src/rust/library/rustc-std-workspace-core)
     Compiling compiler_builtins v0.1.35
     Compiling spirv-std v0.1.0 (/Users/christofer/dev/rust-gpu/fork/crates/spirv-std)
     Compiling glam v0.9.5 (https://github.com/EmbarkStudios/glam-rs?rev=d55ff75e91aab01c4fd9808e3ca9c079dfa7315c#d55ff75e)
     Compiling proc-macro-shader v0.1.0 (/Users/christofer/dev/rust-gpu/fork/examples/shaders/proc-macro-shader)
  error[E0463]: can't find crate for `unroll`
   --> examples/shaders/proc-macro-shader/src/lib.rs:9:3
    |
  9 | #[unroll::unroll_for_loops]
    |   ^^^^^^ can't find crate

  error: aborting due to previous error

  For more information about this error, try `rustc --explain E0463`.
  error: could not compile `proc-macro-shader`

I'm not acquainted enough with the rustc codebase to diagnose the error any further on my own. Hopefully someone else will be able to pick this up :)

[bjorn3]

The code is at https://github.com/rust-lang/rust/blob/ffe52882ed79be67344dd6085559e308241e7f60/compiler/rustc_metadata/src/locator.rs It seems that rustc does some pre-filtering to check if the filename makes any sense. There doesn't seem to be a special case for proc-macros to use the host prefix/suffix. I think it is luck that Windows -> Linux cross-compilation works when using proc-macros. (Or does it?)

[eddyb]

There is a special case here: https://github.com/rust-lang/rust/blob/ae6aa22cf26fede2177abe4ff974030058885b7a/compiler/rustc_metadata/src/creader.rs#L419-L430

        // Load the proc macro crate for the host

        locator.reset();
        locator.is_proc_macro = Some(true);
        locator.target = &self.sess.host;
        locator.triple = TargetTriple::from_triple(config::host_triple());
        locator.filesearch = self.sess.host_filesearch(path_kind);

        let host_result = match self.load(locator)? {
            Some(host_result) => host_result,
            None => return Ok(None),
        };

So all of the self.target in rustc_metadata::locator should refer to the host "target spec".
This does make sense, as e.g. wasm has a similar requirement, and AFAIK proc macros work with it just fine.

But loading a proc macro crate is attempted only if loading failed as a target crate: https://github.com/rust-lang/rust/blob/ae6aa22cf26fede2177abe4ff974030058885b7a/compiler/rustc_metadata/src/creader.rs#L496-L502

            match self.load(&mut locator)? {
                Some(res) => (res, None),
                None => {
                    dep_kind = CrateDepKind::MacrosOnly;
                    match self.load_proc_macro(&mut locator, path_kind)? {
                        Some(res) => res,
                        None => return Err(locator.into_error()),

AFAICT, that .into_error() at the end is the only way to get the error in the issue description, so that means load_proc_macro also fails (with Ok(None) though, not Err), and that's the only reason you're seeing the error.

I'll see what I can get by debugging the relevant part of rustc_metadata.

EDIT: silent loading failure cause found, see #268.