crossSystem results in failed componentResolution

[psionic-k]

I'm developing a cross compiling example and fixing up build & host offsets for the various build cases such as test.

You can see my cargo2nix branch here:
cargo2nix/cargo2nix#214

/examples/3-cross-compiling is the flake of interest. The overlay in /overlays is used to consume the /examples/3-cross-compiling/Cargo.nix in that example's flake file.

I can configure oxalica to give me a Rust toolchain capable of outputting wasm32-wasi, failing only when the stdenv lacks any final wasi linking capability.

To get such a stdenv, I should configure a crossPlatform. However, when I do so, this overlay errors out with a failure to resolve components:

error: Component resolution failed for rust-minimal-1.56.1
       - Target `wasm32-unknown-wasi` is not supported by any components or extensions
       - note: all extensions are: cargo clippy clippy-preview llvm-tools-preview miri miri-preview reproducible-artifacts rls rls-preview rust-analysis rust-analyzer-preview rust-docs rust-mingw rust-src rust-std rustc rustc-dev rustc-docs rustfmt rustfmt-preview
       - note: extensions available for x86_64-unknown-linux-gnu are: cargo clippy clippy-preview llvm-tools-preview reproducible-artifacts rls rls-preview rust-analysis rust-docs rust-std rustc rustc-dev rustc-docs rustfmt rustfmt-preview
       - note: extensions available for wasm32-unknown-wasi are: <empty>
       - note: extensions available for wasm32-wasi are: rust-analysis rust-std
       - note: Check here to see whether a component is available for rustup:
                 https://rust-lang.github.io/rustup-components-history



       … while evaluating 'resolveComponents'

I suspect that instead of host & target, the overlay should be using build & host. This hunk here using target caught my attention:

rust-overlay/rust-overlay.nix

Lines 99 to 118 in f0a3769

	getComponentsWithFixedPlatform = pkgs: pkgname: stdenv:
	let
	pkg = pkgs.${pkgname};
	srcInfo = pkg.target.${super.rust.toRustTarget stdenv.targetPlatform} or pkg.target."*";
	components = srcInfo.components or [];
	componentNamesList =
	builtins.map (pkg: pkg.pkg) (builtins.filter (pkg: (pkg.target != "*")) components);
	in
	componentNamesList;
	getExtensions = pkgs: pkgname: stdenv:
	let
	inherit (super.lib) unique;
	pkg = pkgs.${pkgname};
	rustTarget = super.rust.toRustTarget stdenv.targetPlatform;
	srcInfo = pkg.target.${rustTarget} or pkg.target."*" or (throw "${pkgname} is no available");
	extensions = srcInfo.extensions or [];
	extensionNamesList = unique (builtins.map (pkg: pkg.pkg) extensions);
	in
	extensionNamesList;

My current belief is that target is never used in nixpkgs except when crossing to make a compiler for a third platform, a rare case that can be abstracted out as implementation details for bootstrapping onto a new platform. Rust seems to use host-target terminology. Nix views these offsets as build-host. Backing up by one offset to build-host seems correct. Translation to Rust target triples would be unaffected.

If I'm wrong about this, it will undermine my entire understanding of cross compilation, so I am eager to get to the truth 🐱

At present, everything is working fine for explicit or crossSystem configured targets like aarch64-unknonwn-linux-gnu which have lots of Rust components available. Only when targeting wasi, which only has a rust-std available for that target, do I get these component resolution errors.

[oxalica]

error: Component resolution failed for rust-minimal-1.56.1
       - Target `wasm32-unknown-wasi` is not supported by any components or extensions
...

The error message is not quite clear here. It just failed to find any components with target wasm32-unknown-wasi, because Rust actually provides wasm32-wasi. It's an issue of toRustTarget from nixpkgs.

nix-repl> rust.toRustTarget (lib.systems.elaborate "wasm32-wasi")
"wasm32-unknown-wasi"

We actually hit the toRustTarget issue several times (#52). I'm now considering re-implementing it in rust-overlay so we won't need to rely on and wait for nixpkgs fixes anymore.

I suspect that instead of host & target, the overlay should be using build & host.

No. Note that the overlay provides compiler itself, and the compiler is expected to be used in nativeBuildInputs (offset -1). So compiler's hostPlatform (where compiler running) is your buildPlatform, and compiler's targetPlatform (what kind of ELF compiler produces) is your hostPlatform.

[psionic-k]

We actually hit the toRustTarget issue several times (#52). I'm now considering re-implementing it in rust-overlay so we won't need to rely on and wait for nixpkgs fixes anymore.

I would also rely on the rust-overlay in that case. Cargo2nix is duplicating the function unnecessarily (and likely even worse)

[oxalica]

I just opened the PR NixOS/nixpkgs#146274 to nixpkgs.
But it seems wasm32-wasi still has some evaluation errors for the linker, which is required for rustc. I'll look into it tomorrow.

[oxalica]

Fixed in 7d10573

I added an example about how to cross compile to wasm32-wasi.

[psionic-k]

Beautiful