Trying to cross-compile with buildRustPackage from macOS fails.

[peacememories]

Describe the bug

When trying to build a rust package on macOS with Apple Silicon for other platforms, the rust compiler fails to build.

Steps To Reproduce

Steps to reproduce the behavior:

1. Create a blank rust project with cargo init

2. Add the following default.nix file

   {pkgs ? import <nixpkgs> {}}: let
     pkgsCross = pkgs.pkgsCross.gnu64;
     buildRustPackage = pkgsCross.rustPlatform.buildRustPackage;
   in
     buildRustPackage {
       pname = "test";
       name = "test";
       src = ./.;
       cargoLock = {lockFile = ./Cargo.lock;};
     }

3. Try to run nix-build

4. Observe the error log

Expected behavior

A cross compiled binary should be created

Additional context

Since this section mentions that the rust compiler does not actually have to be built for cross compiling, I tried a different approach:

{pkgs ? import <nixpkgs> {}}: let
  buildRustPackage = pkgs.rustPlatform.buildRustPackage;
in
  buildRustPackage {
    pname = "test";
    name = "test";
    src = ./.;
    cargoLock = {lockFile = ./Cargo.lock;};
    target = "x86_64-unknown-linux-gnu";
  }

Using this method the build is successful, but it results in a macOS binary rather than a cross compiled binary. This is because cargo is still run with --target aarch64-apple-darwin

Notify maintainers

@figsoda (extracted from the blame of buildRustPackage, I don't know how to find the maintainer of a function)

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-darwin"`
 - host os: `Darwin 22.6.0, macOS 13.5`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.13.5`
 - channels(gabriel): `"darwin, home-manager-23.05.tar.gz"`
 - channels(root): `"nixpkgs-23.05-darwin"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

[figsoda]

@NixOS/Rust

[figsoda]

Since this section mentions that the rust compiler does not actually have to be built for cross compiling, I tried a different approach:

{pkgs ? import <nixpkgs> {}}: let
  buildRustPackage = pkgs.rustPlatform.buildRustPackage;
in
  buildRustPackage {
    pname = "test";
    name = "test";
    src = ./.;
    cargoLock = {lockFile = ./Cargo.lock;};
    target = "x86_64-unknown-linux-gnu";
  }

Using this method the build is successful, but it results in a macOS binary rather than a cross compiled binary. This is because cargo is still run with --target aarch64-apple-darwin

This doesn't work because buildRustPackage doesn't look at the target option, but instead always uses the target based on its stdenv. If you prefer to not use nixpkgs's cross compilation, using fenix with naersk or crane could be helpful (fenix's readme has an example of cross compilation with naersk).

[peacememories]

I would honestly prefer being able to just use buildRustPackage with pkgsCross, it just doesn't seem to work for me right now, as noted above.
Since creating the issue I've also tried cargo2nix and crane and have run into similar but different problems.

Cargo2nix works on a project without dependencies, but when trying to build the project I was originally trying to build, it fails on building memchr because it can't find iconv

crane fails with ld: unknown option: --as-needed

here are my current notes on this. They're a bit sparse, I know 😅

[peacememories]

Update - I tried fenix and naersk for cross compiling and I've gotten further, but now I'm stuck again. The project I am building depends on libsqlite3, and therefore fails with /nix/store/l358blr4dfgb3arympc0a88n531f6pj4-x86_64-unknown-linux-musl-binutils-2.40/bin/x86_64-unknown-linux-musl-ld: cannot find -lsqlite3: No such file or directory.
I tried adding buildInputs = [pkgs.pkgsCross.musl64.sqlite]; to the naersk definition, but this does not seem to help.

Full default.nix

{pkgs ? import <nixpkgs> {}}: let
  fenix = builtins.getFlake "github:nix-community/fenix/monthly";
  naersk = builtins.getFlake "github:nix-community/naersk";
  target = "x86_64-unknown-linux-musl";
  toolchain = with fenix.packages.${builtins.currentSystem};
    combine [
      minimal.cargo
      minimal.rustc
      targets.${target}.latest.rust-std
    ];
in
  (naersk.lib.${builtins.currentSystem}.override {
    cargo = toolchain;
    rustc = toolchain;
  })
  .buildPackage {
    src = ./.;
    buildInputs = [
      pkgs.pkgsCross.musl64.sqlite
    ];
    CARGO_BUILD_TARGET = target;
    CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER = let
      inherit (pkgs.pkgsCross.musl64.stdenv) cc;
    in "${cc}/bin/${cc.targetPrefix}cc";
  }

Of course I'd love to get buildRustPackage working fully, but in the meantime getting the naersk workflow running would be very helpful.

[peacememories]

I've since simplified my problems with buildRustPackage to the following:

• nix-build pkgsCross.gnu64.buildPackages.rustc

  Fails with unknown option: --version-script full error log

• nix-build pkgsCross.musl64.buildPackages.rustc

  Fails with couldn't find libc.a in musl libdir: /nix/store/hg3vg4b2r1zgfcilx9l3q0gwi5cpcl3y-libSystem-11.0.0/lib' full error log

These are the same errors that I get while trying to build my rust package

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/cross-compile-rust-from-m1/27879/27

[peacememories]

I think I will close this issue in favor of two issues describing the specific build failures I am experiencing with musl64 and gnu64 while cross compiling rustc.

[alper]

@peacememories Can you link up those issues? I'm also still interested in tracking this (though I've given up myself).

[peacememories]

@alper while trying to isolate this issue I took a bit of a detour. Right now it's failing at #251271 for me