New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Rust cross compilation #50866

Open
wants to merge 1 commit into
base: staging
from

Conversation

Projects
None yet
5 participants
@Mic92
Contributor

Mic92 commented Nov 21, 2018

Status

  • rustc
  • fix non-cross rustc
  • buildRustPackage
  • some packages need fixes & testing
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@Mic92 Mic92 requested a review from LnL7 as a code owner Nov 21, 2018

@Mic92

This comment has been minimized.

Contributor

Mic92 commented Nov 21, 2018

Still compiling. Cross-compiled LLVM is also still broken.

@Mic92 Mic92 changed the title from Rust cross compilation to [WIP] Rust cross compilation Nov 21, 2018

@GrahamcOfBorg

This comment has been minimized.

GrahamcOfBorg commented Nov 21, 2018

Success on x86_64-linux (full log)

Attempted: libpfm, openssl

Partial log (click to expand)

gzipping man pages under /nix/store/m9izkl3p2hr6xngm529jl1gy0ql3gj42-openssl-1.0.2p-man/share/man/
patching script interpreter paths in /nix/store/m9izkl3p2hr6xngm529jl1gy0ql3gj42-openssl-1.0.2p-man
checking for references to /build/ in /nix/store/m9izkl3p2hr6xngm529jl1gy0ql3gj42-openssl-1.0.2p-man...
shrinking RPATHs of ELF executables and libraries in /nix/store/q66rhjlqjkn194c6zji5sjmr3p6z1z6y-openssl-1.0.2p-debug
patching script interpreter paths in /nix/store/q66rhjlqjkn194c6zji5sjmr3p6z1z6y-openssl-1.0.2p-debug
checking for references to /build/ in /nix/store/q66rhjlqjkn194c6zji5sjmr3p6z1z6y-openssl-1.0.2p-debug...
cannot link '/nix/store/.links/0jv8zns4raaa2vc5gjwxqm9n5wx9qi6lbv8556dg6zg80aw3z0yr' to '/nix/store/m9izkl3p2hr6xngm529jl1gy0ql3gj42-openssl-1.0.2p-man/share/man/man3/EVP_PKEY_derive_set_peer.3.gz': No space left on device
cannot link '/nix/store/.links/0jv8zns4raaa2vc5gjwxqm9n5wx9qi6lbv8556dg6zg80aw3z0yr' to '/nix/store/m9izkl3p2hr6xngm529jl1gy0ql3gj42-openssl-1.0.2p-man/share/man/man3/EVP_PKEY_derive_init.3.gz': No space left on device
/nix/store/k4dhk7hqqy3kzmgdvs0j5pi88gfjr4pp-libpfm-4.10.1
/nix/store/pl3km7jksa9r6zrvrwsh5djaxgk004s0-openssl-1.0.2p-bin

@GrahamcOfBorg

This comment has been minimized.

GrahamcOfBorg commented Nov 21, 2018

Success on aarch64-linux (full log)

Attempted: libpfm, openssl

Partial log (click to expand)

checking for references to /build/ in /nix/store/818azw7kbva6jznsj438h05lh33r83wv-openssl-1.0.2p...
shrinking RPATHs of ELF executables and libraries in /nix/store/995mh5wj5dayr041hjldyw2a9wp94qr5-openssl-1.0.2p-man
gzipping man pages under /nix/store/995mh5wj5dayr041hjldyw2a9wp94qr5-openssl-1.0.2p-man/share/man/
patching script interpreter paths in /nix/store/995mh5wj5dayr041hjldyw2a9wp94qr5-openssl-1.0.2p-man
checking for references to /build/ in /nix/store/995mh5wj5dayr041hjldyw2a9wp94qr5-openssl-1.0.2p-man...
shrinking RPATHs of ELF executables and libraries in /nix/store/8arzdc863bwrqx50jv441vqfxfbxpjgg-openssl-1.0.2p-debug
patching script interpreter paths in /nix/store/8arzdc863bwrqx50jv441vqfxfbxpjgg-openssl-1.0.2p-debug
checking for references to /build/ in /nix/store/8arzdc863bwrqx50jv441vqfxfbxpjgg-openssl-1.0.2p-debug...
/nix/store/d8g0sv493ar86xvz97bc6ndlv6h3l5g4-libpfm-4.10.1
/nix/store/7y00xv5kwkjvy3pq4npq6finarpyxwc5-openssl-1.0.2p-bin

nativeBuildInputs = [unzip];
propagatedNativeBuildInputs = [ findXMLCatalogs ];
nativeBuildInputs = [ unzip ];
propagatedBuildInputs = [ findXMLCatalogs ];

This comment has been minimized.

@matthewbauer

matthewbauer Nov 21, 2018

Member

This is a actually a bug I think. "propagatedNativeBuildInputs" should be correct. Will have a fix soon hopefully.

This comment has been minimized.

@Mic92

Mic92 Nov 21, 2018

Contributor

I guess I can wait a bit, I just get the things working properly and not blocking on nitpicks.

This comment has been minimized.

@Ericson2314

Ericson2314 Nov 28, 2018

Member

findXMLCatalogs does sound like a tool that should be propagatedNativeBuildInputs.

@@ -105,6 +105,10 @@ let
'';
postFixup = ''
# fixes cross-compilation to pick the right perl,
# since perl is also in nativeBuildInputs
(sed -i -e '1 s!^.*$!${perl}/bin/perl!' $bin/bin/c_rehash)

This comment has been minimized.

@matthewbauer

matthewbauer Nov 21, 2018

Member

I didn't think this would be a problem now that #49608 is merged. Do you know where this comes from? It also looks like openssl supports passing in HASHBANGPERL to do this explicity.

This comment has been minimized.

@Mic92

Mic92 Nov 21, 2018

Contributor

HASHBANGPERL is only supported in openssl 1.1, but we still use openssl 1.0.2 in most places.

This comment has been minimized.

@Mic92

Mic92 Nov 21, 2018

Contributor

Also #49608 was reverted again in staging by @LnL7

This comment has been minimized.

@LnL7

LnL7 Nov 21, 2018

Contributor

Yeah, the stdenv was broken. Since this is the second time I think that change has a bit too much impact to merge into staging directly, we should run it through a separate jobset first to get an overview of the breakages.

This comment has been minimized.

@Mic92

Mic92 Nov 21, 2018

Contributor

I probably don't need this change to make cross-compiling rust packages work. So I will split it of probably later.

@GrahamcOfBorg

This comment has been minimized.

GrahamcOfBorg commented Nov 21, 2018

Timed out, unknown build status on x86_64-darwin (full log)

Attempted: openssl

The following builds were skipped because they don't evaluate on x86_64-darwin: libpfm

Partial log (click to expand)

ld: warning: directory not found for option '-L../bootstrap/x86_64-apple-darwin/usr/lib'
ld: warning: undefined base symbol '__TMC15SwiftFoundation19_NSCFConstantString' for alias '___CFConstantStringClassReference'
installing
post-installation fixup
strip is /nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools/bin/strip
patching script interpreter paths in /nix/store/awcr3pzcd5vpqwh3lhb0ijxbv72vf9l7-swift-corefoundation
cannot build derivation '/nix/store/4xqfqm5lbdqpb9qds1xxrzmyss3818h3-stdenv-darwin.drv': 7 dependencies couldn't be built
cannot build derivation '/nix/store/mv8zixg5sy9lqnqnn15ca4wl2vxipii6-perl-5.28.0.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/7lipgdn8jvyy7k45idq1sh17k7bpj9z3-openssl-1.0.2p.drv': 2 dependencies couldn't be built
error: build of '/nix/store/7lipgdn8jvyy7k45idq1sh17k7bpj9z3-openssl-1.0.2p.drv' failed

@Mic92

This comment has been minimized.

Contributor

Mic92 commented Nov 21, 2018

Mhm, I currently stuck at this fix-point stuff that is used to create buildRustPackage. It somehow puts a non-native rustc/cargo into nativeBuildInputs:

nix-repl> pkgsCross.aarch64-multiplatform.fd.nativeBuildInputs
[ «derivation /nix/store/d9abfxjb0icbjg7k8j62pasgdj7s8zwl-git-2.19.1.drv» «derivation /nix/store/gnvkwv8mls6a2n4bpw3xdpp6fh1d1lyc-cargo-1.30.0-aarch64-unknown-linux-gnu.drv» «derivation /nix/store/hibvax8p73n9imhi9ix7z55ka5hl47ai-rustc-1.30.1-aarch64-unknown-linux-gnu.drv» ]

@Mic92 Mic92 force-pushed the Mic92:rust-cross branch from 45d9f1d to 9723ba9 Nov 21, 2018

@Mic92

This comment has been minimized.

Contributor

Mic92 commented Nov 21, 2018

Build inputs are clean now.

$ nix-repl> pkgsCross.aarch64-multiplatform.fd.nativeBuildInputs
[ «derivation /nix/store/zp9aqlm1gg9k2hz990yvvwfgja3ybyiy-cargo-1.30.0.drv» «derivation /nix/store/4sjcxw65jn8l90ql2ivn2znlpw24fc09-rustc-1.30.1.drv» «derivation /nix/store/d9abfxjb0icbjg7k8j62pasgdj7s8zwl-git-2.19.1.drv» «derivation /nix/store/8ns0q8dc8bcxyc178ck6zcmhp39rvhwa-nss-cacert-3.40.drv» ]

I think most cross-compile fixes from this pr are actually not needed.
I will separate them later into a new pull request.

@Mic92 Mic92 force-pushed the Mic92:rust-cross branch from 9723ba9 to c37efea Nov 25, 2018

@Mic92 Mic92 force-pushed the Mic92:rust-cross branch from 00f0bc7 to 97d2e9b Nov 25, 2018

@Mic92

This comment has been minimized.

Contributor

Mic92 commented Nov 25, 2018

Compilation works now!

$ rustc --target=aarch64-unknown-linux-gnu foo.rs
$ file foo
foo: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/1r1pdrdwgj6k3wy3bhrxjiqjdxf9z2f5-glibc-2.27-aarch64-unknown-linux-gnu/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, with debug_info, not stripped

@Mic92 Mic92 force-pushed the Mic92:rust-cross branch from 97d2e9b to 4116129 Nov 25, 2018

@Mic92 Mic92 referenced this pull request Nov 25, 2018

Merged

Call for Content: 2018/13 #71

@Mic92 Mic92 force-pushed the Mic92:rust-cross branch from 4116129 to 7a7313e Nov 26, 2018

@Mic92 Mic92 force-pushed the Mic92:rust-cross branch from af33ed7 to debf6f3 Nov 28, 2018

@Mic92

This comment has been minimized.

Contributor

Mic92 commented Nov 28, 2018

The first working buildRustPackage!

$ file result/bin/fd
result/bin/fd: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/1r1pdrdwgj6k3wy3bhrxjiqjdxf9z2f5-glibc-2.27-aarch64-unknown-linux-gnu/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, not stripped

@Mic92 Mic92 referenced this pull request Nov 28, 2018

Closed

[WIP] Port switch-to-configuration script to Python #48378

0 of 9 tasks complete
"-DTARGET_TRIPLE=${stdenv.hostPlatform.config}"
"-DLLVM_DEFAULT_TARGET_TRIPLE=${stdenv.targetPlatform.config}"
"-DLLVM_TARGETS_TO_BUILD=${llvmTarget stdenv.hostPlatform};${llvmTarget stdenv.targetPlatform}"

This comment has been minimized.

@Ericson2314

Ericson2314 Nov 28, 2018

Member

Can we do this for the other LLVMs?

This comment has been minimized.

@Mic92

Mic92 Nov 29, 2018

Contributor

I will split this off into a different pull request: #51226

@Mic92 Mic92 force-pushed the Mic92:rust-cross branch 2 times, most recently from 86a00de to 6cd6902 Nov 29, 2018

@Ericson2314

This comment has been minimized.

Member

Ericson2314 commented Nov 29, 2018

rustBootstrap should be used as buildPackages.rustBootstrap in (almost?) all cases.

in stdenv.mkDerivation (args // {
inherit cargoDeps;
patchRegistryDeps = ./patch-registry-deps;
buildInputs = [ cacert git cargo rustc ] ++ buildInputs;
nativeBuildInputs = [ cargo rustc git cacert ] ++ nativeBuildInputs;

This comment has been minimized.

@Ericson2314

Ericson2314 Nov 29, 2018

Member

Note these would be buildPackges.cargo buildPackages.rustc, etc, due to splicing black magic.

# For beta and nightly releases use the nixpkgs-mozilla overlay
rust = callPackage ../development/compilers/rust ({
inherit (darwin.apple_sdk.frameworks) CoreFoundation Security;
rustBootstrap = recurseIntoAttrs (makeRustPlatform (callPackage

This comment has been minimized.

@Ericson2314

Ericson2314 Nov 29, 2018

Member

Oh sorry, rather than changing references to rustBootstrap, makeRustPlatform could use buildPackages.callPackage here. I'd have to stare at the details longer, but the gist of it is all the compilers should come from the previous stage (buildPackages).

This comment has been minimized.

@Mic92

Mic92 Dec 9, 2018

Contributor

What is the practical difference of doing so?

@Ericson2314

This comment has been minimized.

Member

Ericson2314 commented Dec 1, 2018

So me flipping on the Rust bootstrap stuff aside, here is the actual requirement.

  • rustc build != host == target
  • buildPackages.rustc build == host != target
  • buildPackages.buildPackages.rustc build == host == target

It is tempting to also have:

-buildPackages.buildPackages.buildPackages.rustc prebuilt build == host == target

But sadly we cannot do that as buildPackages^(n+2) = buildPackages^2, as that is they native packages and they are a fixed point.

Solution would be a rustcBinary derivation with a assert that host and target are the same. (Build platform doesn't matter since just unpacking prebuilt.)

The rust and rustBootstrap "platforms" are like stdenv in that they provide something analogous to mkDerivation. Likewise they should use buildPackages.rustc[Binary] just as stdenv uses buildPackage.cc—tools must be sourced from the previous stage.

Also the general pattern in those tools is fooForXXX = XXXPackages.(buildPackages|stdenv).foo, with hostPackages = pkgs. I.e. no need to every be clear about remapping platforms on the level of individual packages, instead use buildPackages and targetPackages to use a package with the *Platforms slid over.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment