Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shift Nix Platform Offsets Minus One #58

Closed

Conversation

psionic-k
Copy link

Rust uses host-target terminology. From the perspective of serving toolchains,
this makes sense. You are on a build platform and distributing toolchains to
run on hosts and generate outputs for targets.

However, from the perspective of the person downloading a toolchain, they are
seeking a compiler for their buildPlatform to generate outputs for their
hostPlatform, where their Rust program will run.

This change shifts most targets to host and hosts to build.

When providing build inputs for a native or cross build, depsBuildHost are
available during build time to make outputs for the host. Therefore, we
propagate libiconv for linking into the outputs on the host. depsBuildHost is
also known as nativeBuildInputs. propagatedNativeBuildInputs could also be used
here.

https://nixos.org/manual/nixpkgs/stable/#variables-specifying-dependencies

Aiming to fix #57

Currently downloading the gigabyte of deps for the wasi target. Seems to be working. I'll have the cargo2nix branch with the example my version of the overlay up pretty soon.

Rust uses host-target terminology.  From the perspective of serving toolchains,
this makes sense.  You are on a build platform and distributing toolchains to
run on hosts and generate outputs for targets.

However, from the perspective of the person downloading a toolchain, they are
seeking a compiler for their buildPlatform to generate outputs for their
hostPlatform, where their Rust program will run.

This change shifts most targets to host and hosts to build.

When providing build inputs for a native or cross build, depsBuildHost are
available during build time to make outputs for the host.  Therefore, we
propagate libiconv for linking into the outputs on the host.  depsBuildHost is
also known as nativeBuildInputs.  propagatedNativeBuildInputs could also be used
here.

https://nixos.org/manual/nixpkgs/stable/#variables-specifying-dependencies
@oxalica
Copy link
Owner

oxalica commented Nov 16, 2021

Please check #57 (comment)

About the 3 platforms, you can check this wiki: https://nixos.wiki/wiki/Cross_Compiling
In short, for a program, buildPlatform is where it's being built, hostPlatform is where it's running, and targetPlatform is what it produces (only applied to compiler, binutils, etc).

For a compiler (like rustc provided by rust-overlay), only it's hostPlatform and targetPlatform matters to users. Because we only concern about "where compiler is running" and "what the compiler produces". If you are on x86_64 and want to build an aarch64 ELF, you should seek for a compiler whose hostPlatform is x86_64 and targetPlatform is aarch64.

You don't need to calculate this offset thing, since it's automatically done by nixpkgs through package splicing, a magic in callPackage. For common usage, this overlay is introduced by,

import <nixpkgs> {
  overlays = [
    (import <rust-overlay>)
    (final: prev: {
      pkg1 = final.callPackage ./myp-kg.nix {};
      pkg2 = final.callPackage ./my-another-pkg.nix {};
    })
  ];
}

And when we package something, we write,

{ stdenv, rust-bin }:
stdenv.mkDerivation {
  nativeBuildInputs = [ rust-bin.stable.latest.minimal ];
  # ...
}

Or if we want to use buildRustPackage like naersk does,

{ rustPlatform }:
(makeRustPlatform {
  rustc = rust-bin.stable.latest.minimal;
  cargo = rust-bin.stable.latest.minimal;
}).buildRustPackage {
  # ...
}

In both cases, the compiler goes nativeBuildInputs, and it automatically gets a -1 platform offset.
And we can easily use nixpkgs' mechanism to select programs from correct platform, by,

(import <nixpkgs> { crossSystem = "aarch64-linux"; overlays = ...; }).my-pkg

or

(import ...).pkgsCross.riscv64.my-pkg

Then everything just works.

@oxalica
Copy link
Owner

oxalica commented Nov 16, 2021

Also there is a cross compilation example in examples directory.

I believe this PR would break that example and make you must explicit specify rustc's platform every time since it mess up nix's platform selection.

@psionic-k
Copy link
Author

I'm beginning to reconcile the sliding window. I'm going to patch over the nixpkgs toRustTriple so I can go back and finish another pass at cargo2nix's offsets. Would bet money the mkcrate.nix is going to need some patches for wasm & wasi. Thanks for the quick reply on the issue! Saved me time to work on real problems.

@psionic-k psionic-k closed this Nov 16, 2021
@psionic-k psionic-k deleted the minus-one-nix-offset branch November 16, 2021 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

crossSystem results in failed componentResolution
2 participants