When using serde_derive via rules_rust, build fails with: failed to spawn process "No such file or directory"

[kiron1]

When using (transitively) serde_derive in a Bazel workspace via rules_rust, the build fails with:

error: proc-macro derive panicked
 --> main.rs:3:10
  |
3 | #[derive(Serialize, Deserialize, Debug)]
  |          ^^^^^^^^^
  |
  = help: message: failed to spawn process: Os { code: 2, kind: NotFound, message: "No such file or directory" }

This only happens on #[cfg(all(target_arch = "x86_64", target_os = "linux", target_env = "gnu"))].

I think it is related to serde release 1.0.172.

Due to:

• https://github.com/serde-rs/serde/blob/v1.0.172/precompiled/serde_derive/src/lib.rs#L24-L25
• https://github.com/serde-rs/serde/blob/v1.0.172/precompiled/serde_derive/src/lib_precompiled.rs#L32-L40

Minimal reproducible example

WORKSPACE

workspace(name = "rules_rust_serde")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_rust",
    sha256 = "b4e622a36904b5dd2d2211e40008fc473421c8b51c9efca746ab2ecf11dca08e",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_rust/releases/download/0.19.1/rules_rust-v0.19.1.tar.gz",
        "https://github.com/bazelbuild/rules_rust/releases/download/0.19.1/rules_rust-v0.19.1.tar.gz",
    ],
)

load("@rules_rust//rust:repositories.bzl", "rust_repositories")

rust_repositories(
    edition = "2021",
)

load("@rules_rust//crate_universe:repositories.bzl", "crate_universe_dependencies")

crate_universe_dependencies()

load("@rules_rust//crate_universe:defs.bzl", "crate", "crates_repository")

crates_repository(
    name = "crate_index",
    # cargo_config = "//:.cargo/config.toml",
    cargo_lockfile = "//:Cargo.lock",
    # Generate with:
    # CARGO_BAZEL_REPIN=1 bazel sync --only=crate_index
    lockfile = "//:Cargo.Bazel.lock",
    packages = {
        "serde": crate.spec(version = "1.0.174"),
        "serde_derive": crate.spec(version = "1.0.174"),
    },
)

load("@crate_index//:defs.bzl", "crate_repositories")

crate_repositories()

load("@rules_rust//bindgen:repositories.bzl", "rust_bindgen_repositories")

rust_bindgen_repositories()

BUILD

load("@rules_rust//rust:defs.bzl", "rust_binary")

rust_binary(
    name = "main",
    srcs = [
        "main.rs",
    ],
    proc_macro_deps = ["@crate_index//:serde_derive"],
    deps = ["@crate_index//:serde"],
)

main.rs

use serde_derive::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
struct Point {
    x: i32,
    y: i32,
}


fn main() {
    let _point = Point { x: 1, y: 2 };
}

[ttiurani]

I'm hitting this one too. The error might be related to

serde-rs/serde#2516

which was fixed swiftly for Cargo users.

[kiron1]

@ttiurani the issue you linked and the issue I describe here a two very different issues. The fix in https://github.com/serde-rs/serde/releases/tag/v1.0.173 for serde-rs/serde#2516 does not solve the problem described in this issue.

I assume the problem with "No such file or directory", is that crate_universe does not make the binary available which will be used in the proc macro of serde_derive.

[ttiurani]

Yes indeed, should have emphasized the "might" part!

I thought it best to give some context related to that upstream serde issue which I almost commented on before finding this issue, but didn't mean to imply that this rules_rust problem was fixed with that change.

[fmorency]

I'm also getting bitten by this issue!

[yuanweixin]

+1 here.

[kolloch]

For what is worth, you can work around this issue by locking an earlier version:

serde = { version = "=1.0.164", features = ["derive"] }

https://github.com/kolloch/rust-bazel-missing-proc-macro2/blob/main/Cargo.toml#L8-L9

[Thank you for the great bug report BTW, I was just assembling mine and found this after but honestly had nothing to add :) ]

[ibigbug]

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
error: proc-macro derive panicked
  --> external/crate_index__maxminddb-0.23.0/src/maxminddb/lib.rs:67:10
   |
67 | #[derive(Deserialize, Debug)]
   |          ^^^^^^^^^^^
   |
   = help: message: file missing from serde_derive manifest directory during macro expansion: /home/runner/.cache/bazel/_bazel_runner/6b56f1ca6533467c2b39241ad2017cd4/sandbox/linux-sandbox/551/execroot/clash-rs/external/crate_index__serde_derive-1.0.178/serde_derive-x86_64-unknown-linux-gnu

error: proc-macro derive panicked
 --> external/crate_index__maxminddb-0.23.0/src/maxminddb/geoip2.rs:4:10
  |
4 | #[derive(Deserialize, Serialize, Clone, Debug)]
  |          ^^^^^^^^^^^

I'm getting this instead, not sure if it's same thing

[kiron1]

The error message changed since version 1.0.176, see: https://github.com/serde-rs/serde/blob/v1.0.176/precompiled/serde_derive/src/lib_precompiled.rs#L44-L52 but the root cause of the problem remains the same. See also #2074

[ibigbug]

I'd like to know if there's a way to hijack the dependency of a 3rd party crate from crates_repository so this can be totally workaround but I couldn't find a way to

[ibigbug]

Hi, I see that #2097 is merged - I have a dump question - what do I need to do to make the build fixed? ty

[ttiurani]

Hi, I see that #2097 is merged - I have a dump question - what do I need to do to make the build fixed? ty

I tried to replicate the bug with a bootstrapped version of rules_rust here:

https://github.com/ttiurani/rules_rust/tree/main/examples/crate_universe_failure/serde_derive

and unfortunately the merge of that PR alone (at least as far as I can tell) did not seem to fix the problem.

[ttiurani]

Good news: my tester now compiles again with serde_derive v.1.0.184 and later. This is because serde removed prebuilt binaries:

https://github.com/serde-rs/serde/releases/tag/v1.0.184

Things possibly again work also with older versions of rules_rust?

Edit. At least for my builds with a new serde version, this error is gone. I didn't update rules_rust either.

[ibigbug]

oh wow

[kiron1]

With the recent update of serde to not use the ore compiled binary anymore, this issue became outdated.