Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc: Improve musl support #105383

Merged
merged 1 commit into from Dec 2, 2020
Merged

rustc: Improve musl support #105383

merged 1 commit into from Dec 2, 2020

Conversation

@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Nov 29, 2020

Motivation for this change

There was a slight error in the target logic I didn't notice before, and
also should do the same thing for the other platforms.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@lopsided98
Copy link
Contributor

@lopsided98 lopsided98 commented Nov 29, 2020

I'd like to understand this better; could you explain what the relevant difference is here?

Also, you can remove the targetPackages argument I added.

Mic92
Mic92 approved these changes Nov 29, 2020
@Mic92
Copy link
Member

@Mic92 Mic92 commented Nov 29, 2020

I'd like to understand this better; could you explain what the relevant difference is here?

Also, you can remove the targetPackages argument I added.

When you compile stuff than any or all of the following platforms can be musl: build (the platform the build is running on), host (the platform the binary is built for) or target (the platform the compiler can compile code for). This is now correctly set by the expression.

@lopsided98
Copy link
Contributor

@lopsided98 lopsided98 commented Nov 29, 2020

I meant the difference between targetPackages and pkgsBuildTarget.targetPackages.

@lopsided98
Copy link
Contributor

@lopsided98 lopsided98 commented Nov 29, 2020

I tried building pkgsStatic.rustc to test the host platform part, but it looks like LLVM would need some fixes first:

  /nix/store/w5ywkv7ph4zji8lg46640p39042ajxw2-x86_64-unknown-linux-musl-binutils-2.31.1/bin/x86_64-unknown-linux-musl-ld: /nix/store/jgrasq40yhnvarrscqxyav98773mf5ib-x86_64-unknown-linux-musl-stage-final-gcc-debug-9.3.0/lib/gcc/x86_64-unknown-linux-musl/9.3.0/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
  /nix/store/w5ywkv7ph4zji8lg46640p39042ajxw2-x86_64-unknown-linux-musl-binutils-2.31.1/bin/x86_64-unknown-linux-musl-ld: /nix/store/jgrasq40yhnvarrscqxyav98773mf5ib-x86_64-unknown-linux-musl-stage-final-gcc-debug-9.3.0/lib/gcc/x86_64-unknown-linux-musl/9.3.0/crtend.o: relocation R_X86_64_32 against `.ctors' can not be used when making a shared object; recompile with -fPIC
  /nix/store/w5ywkv7ph4zji8lg46640p39042ajxw2-x86_64-unknown-linux-musl-binutils-2.31.1/bin/x86_64-unknown-linux-musl-ld: final link failed: nonrepresentable section on output
  collect2: error: ld returned 1 exit status
  make[2]: *** [tools/polly/lib/CMakeFiles/LLVMPolly.dir/build.make:196: lib/LLVMPolly.so] Error 1
  make[1]: *** [CMakeFiles/Makefile2:31037: tools/polly/lib/CMakeFiles/LLVMPolly.dir/all] Error 2
  make: *** [Makefile:171: all] Error 2
builder for '/nix/store/1dkhrzniv1lvpza62jm6d0q0xbkinzjh-llvm-10.0.1-x86_64-unknown-linux-musl.drv' failed with exit code 1
cannot build derivation '/nix/store/rjswaxfhn3f37cifl2kxd89rr4asrsbv-rustc-1.47.0-x86_64-unknown-linux-musl.drv': 1 dependencies couldn't be built

There was a slight error in the target logic I didn't notice before, and
also should do the same thing for the other platforms.
@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Dec 2, 2020

I meant the difference between targetPackages and pkgsBuildTarget.targetPackages.

  1. buildPackages and targetPackages are semi-inverses, in that x.buildPackages.targetPackages = x, and x.targetPackages.buildPackages = x if x has a targetPackages (during cross the last stage doesn't)
  2. buildPackages == pkgsBuildHost, targetPackages -= pkgsTargetTarget
  3. stdenv.cc is a package drawn from buildPackages, like buildPackages.gcc or buildPackages.clang
  4. <some-cc>.libc is a package drawn from targetPackages, like glibc (or glibcCross, because GCC bootstrapping is the worst).
    4 pkgsXxYy.targetPackages = pkgsYyZz for some Zz (it depends)

Putting it altogether:

  1. pkgsXxYy.targetPackages.stdenv.cc = pkgsXxYy.targetPackages.buildPackages.<some-cc> = pkgsXxYy.<some-cc>.
  2. pkgsXxYy.<some-cc>.libc = pkgsXxYy.targetPackges.<some-libc> = pkgsYyZz.<some-libc>

pkgsBuildYy.<some-cc> means a tool we can run in this derivation, targetting Yy. This is what we want!
pkgsYyZz.<some-libc> means a library that runs on Yy, which is what we want! Zz is not determined but that's OK because the target platform of libc is ignored/meaningless.

@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Dec 2, 2020

I tried building pkgsStatic.rustc to test the host platform part, but it looks like LLVM would need some fixes first

To be clear (not trying to imply you disagree) I still like doing these fixes even if they are of no practical benefit because:

  1. If/when the dependencies are fixed, stuff should just work
  2. People very often base new packages off existing packages, so I want existing package's source code to be the best model of how things are done it can be.

@Ericson2314 Ericson2314 merged commit 68fa053 into master Dec 2, 2020
18 of 19 checks passed
@Ericson2314 Ericson2314 deleted the more-rustc-musl branch Dec 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants