Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: treewide: Always prefix compilers #44583

Closed
wants to merge 13 commits into from

Conversation

@Ericson2314
Copy link
Member

Ericson2314 commented Aug 7, 2018

Motivation for this change

The goal is to bring the native and cross code paths closer together by always prefixing compilers. This makes it easier to catch programs that hard-code gcc or clang.

This also includes:

  • Two patches to help with darwin -> linux cross compilation; since they're fine patches and the same override-gcc fix is needed.
  • Making the wrapper env vars define CC, LD, and friends use full paths instead of names, generally good for robustness.

The whole thing I plan to merge after 18.09, but maybe those 2 bits can go in first.

Fixes #21471

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

Ericson2314 and others added 12 commits Jul 3, 2018
Then we don't run the risk of stomping over non-autotools.
This isn't needed now that we use env vars.
GNU tools are prefixed the same whether wrapped or not, but unwrapped
LLVM tools are not prefixed because they are multi-target.
It's confusing when the linker and assembler appear to come from
cc-wrapper rather than bintools-wrapper.
This isn't a MUSL thing, but just needed for cross compilation to x86.
No one had tried this when all cross compilation was to linux + glibc,
hence why no one noticed this until recently.
Carefully fake cc-version and cc-fullversion to avoid needing a compiler
for the kernel itself to build the headers.
The unwrapped bintools are already propagated dependencies, and therefore on the
PATH of downstream builds, cc-wrapper can get the
assembler on its own.
@Ericson2314 Ericson2314 force-pushed the obsidiansystems:always-prefix branch to 6c60a14 Aug 7, 2018
@LnL7
Copy link
Member

LnL7 commented Aug 7, 2018

I only glanced very quickly over the changes and this wasn't immediately clear, the un-prefixed wrappers are still available and work right?

sed "s|--owner 0 --group 0||g" -i Makefile
sed -i Makefile \
-e 's|--owner 0 --group 0||g' \
-e '/CC:=gcc/d'

This comment has been minimized.

Copy link
@Mic92

Mic92 Aug 7, 2018

Contributor

Do we have any use for paxctl/grsecurity at all? We removed the kernel long time ago.

This comment has been minimized.

Copy link
@Ericson2314

Ericson2314 Aug 7, 2018

Author Member

No I don't. But it's part of bootstrapping; that's why I fixed it.

@Ericson2314
Copy link
Member Author

Ericson2314 commented Aug 7, 2018

@LnL7 actually wrapped binaries are always prefixed with this PR. But I could make targetPrefix a parameter. inPrefix is added because GCC will be prefixed on the inside but Clang won't be, but yes they can be varied independently in principle.

@dezgeg
Copy link
Contributor

dezgeg commented Aug 7, 2018

NAK for removing unprefixed gcc, clang from native compilation. stdenv stands for "standard environment", and that's what it should stay, standard.


# Skip clean on darwin, case-sensitivity issues.
buildPhase = lib.optionalString (!buildPlatform.isDarwin) ''
make mrproper $makeFlags

This comment has been minimized.

Copy link
@matthewbauer

matthewbauer Aug 7, 2018

Member

Can you use buildFlags & checkFlags here instead?

@matthewbauer
Copy link
Member

matthewbauer commented Aug 8, 2018

Can we have an unprefixed symlink for the host compiler? Even just something like:

ln -s $CC $out/bin/cc
ln -s $CXX $out/bin/c++

Would cover most use cases. Maybe one for gcc as well because that's often hardcoded.

@Ekleog
Copy link
Member

Ekleog commented Oct 3, 2018

I completely agree with the gain offered by always prefixing the compilers in stdenv: it is the removal of another source of impurity (the host system), which is what nix is all about.

However, a cursory read (which can easily be wrong) makes me think nix-shell -p gcc would also only yield prefixed-gcc, which would not be acceptable as gcc is then supposed to be used by the end-user, and not by nix. Did I read correctly?

@Ericson2314
Copy link
Member Author

Ericson2314 commented Oct 3, 2018

gcc and stdenv.cc are currently the same thing (for Linux). I didn't change that but we could.

I sort of rather fix that after as cc-wrapper has a bunch of other issues for manual usage too.

@mmahut
Copy link
Member

mmahut commented Aug 19, 2019

Are there any updates on this pull request, please?

@matthewbauer matthewbauer mentioned this pull request Sep 12, 2019
4 of 10 tasks complete
@cdepillabout
Copy link
Member

cdepillabout commented May 5, 2020

It looks like there hasn't been any updates on this in a year, so I will close it.

(@Ericson2314 sorry in advance if you're still working on any of these PRs that I've closed. Definitely don't hesitate to re-open. I've been trying to clean up old PRs tagged haskell.)

@Ericson2314
Copy link
Member Author

Ericson2314 commented May 5, 2020

@cdepillabout It's fine. I'll reopen if/when I get back to them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

9 participants
You can’t perform that action at this time.