Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCC has unwanted flags #18995

Open
andrewrk opened this issue Sep 27, 2016 · 93 comments · Fixed by #28029
Open

GCC has unwanted flags #18995

andrewrk opened this issue Sep 27, 2016 · 93 comments · Fixed by #28029
Assignees
Labels
0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 1.severity: blocker This is preventing another PR or issue from being completed
Milestone

Comments

@andrewrk
Copy link
Member

andy@xps:~/tmp$ NIX_DEBUG=true gcc
(some good flags for dealing with /nix/store/* and then...)
  -O2
  -D_FORTIFY_SOURCE=2
  -fstack-protector-strong
  --param
  ssp-buffer-size=4
  -fPIC
  -fno-strict-overflow
  -Wformat
  -Wformat-security
  -Werror=format-security

These flags should be enabled when compiling nix packages, but they should not be always on for general compiler use.

This has caused a broken debugging experience for me on the project I was working on.

Most projects have a "debug build" that has optimizations off, and this prevents that from working correctly.

NixOS version 16.09pre90254.6b20d5b (Flounder)

@andrewrk
Copy link
Member Author

I believe this was caused by #12895

@andrewrk
Copy link
Member Author

Not only that, but these flags are in the extraAfter section, which means I can't even override the flags, for example with -Og in a Makefile or build script.

@andrewrk
Copy link
Member Author

One workaround is putting

    hardeningDisable = [ "fortify" ];

in a nix-shell. Since I was already using a nix-shell for my environment, this is an acceptable workaround. Still, seems weird to assert -O2 by default and have to disable it to get previously expected behavior.

@abbradar
Copy link
Member

abbradar commented Sep 27, 2016

I think this can be fixed by adding a special flag, NIX_ENFORCE_HARDENING, that is only set in stdenvs. This way the flags won't be enforced in regular gcc invocations and could be disabled in shells by setting this flag to 0.

BTW having -O2 always added is not nice because some applications might want -O3 or -Ofast (which would be overridden). We may want to make this conditional somehow...

@rasendubi
Copy link
Member

/cc @globin @fpletz

@fpletz
Copy link
Member

fpletz commented Sep 27, 2016

You can use

hardeningDisable = [ "all" ];

to disable all hardening flags. This should also work if set as an environment variable.

We will work on a solution after 16.09 using gcc spec files that can detect for example if libraries are built or debugging is enabled. This was just our first iteration and we agree that it is not perfect. Changes to the cc-wrapper always require a full rebuild, which is very painful. More feedback on #12895 would've been helpful.

We didn't anticipate and test that those flags would be propagated to regular gcc invocations outside of nix builds. Not entirely sure how to fix it yet but @abbradar's proposal sounds reasonable. I will look into it after 16.09.

-O2 is required for -D_FORTIFY_SOURCE=2. Not sure though if higher optimizations also work.

@abbradar
Copy link
Member

FWIW I found that level 1 fortifying requires inlining:

As I said in that post, the special fortified functions (those that are available in the form of __$func_chk in the libc.so file and provide warnings at build time, and proper stack traces at runtime) only get enabled if inline functions are enabled, so are totally ignored at -O0 (simply disabling inlines, by using -fno-inline won’t stop them from being used, though).

I haven't been able to find anything except of anecdotal evidence that -O2 is required for level 2 (I'm not questioning your choice, rather I was trying to find a list of optimizations that are required so that we can enable them individually).

@fpletz
Copy link
Member

fpletz commented Sep 27, 2016

(I'm not questioning your choice, rather I was trying to find a list of optimizations that are required so that we can enable them individually).

I'm always open for suggestions. Thanks! 😃 We'll have to try that.

@abbradar
Copy link
Member

Unfortunately it may need peeking into rat nest GCC/glibc's codebases to determine this. Meanwhile I think it's okay to leave -O2.

@copumpkin
Copy link
Member

Just as a quick reminder when you talk about gcc: there's also clang to consider on Darwin (and in some cases on Linux too)

@jxy
Copy link

jxy commented Sep 29, 2016

So this is why recently gdb suddenly tells me different things in my code and gcc no longer does a good job optimizing my code. I almost thought gcc 5.4 had optimization regressions.

How do I get back a reasonable working gcc? Which nixpkgs version should I revert to?

Does this mean I should put hardeningDisable = [ "all" ]; in all of my custom nix files for my HPC production code, eg, mpich3?

I'm using nixpkgs on a ubuntu without root privilege.

It is NOT okay to leave -Oanything in extraAfter!

@andrewrk
Copy link
Member Author

We're all in agreement here @jxy and going to fix it soon. This is why it's called unstable!

The workaround you mentioned is working for me and is probably the reasonable thing to do until this is fixed.

@jxy
Copy link

jxy commented Sep 29, 2016

Right, thanks for the workaround. It wasn't easy to find this thread. I guess I'll need some performance regression tests after changing my system.

@bjornfor bjornfor added 0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer labels Sep 30, 2016
@jxy
Copy link

jxy commented Oct 3, 2016

Does NixOS 16.09 release version have this issue? If so, I'll put off upgrading until this is fixed.

@joachifm joachifm mentioned this issue Oct 7, 2016
17 tasks
@fpletz fpletz added this to the 16.09 milestone Oct 10, 2016
@fpletz fpletz self-assigned this Oct 10, 2016
@vcunat
Copy link
Member

vcunat commented Oct 11, 2016

Yes, both 16.09 and unstable/master.

@fpletz
Copy link
Member

fpletz commented Oct 11, 2016

I'm currently working on a fix.

@andrewrk
Copy link
Member Author

even though I knew about this, I temporarily forgot and it caused me to file a bogus issue report on another project: thejoshwolfe/legend-of-swarkland#36

@mboisson
Copy link

mboisson commented Feb 3, 2017

I'm running into this issue when compiling older versions of GCC.

gcc 4.8 compiles fine, but at run time, it tries to use stack-protector-strong, which is not supported.

I've wasted about 8 hours so far trying to fix this broken thing....
[mboisson@build-node easybuild-easyconfigs]$ gcc --version
gcc: erreur: unrecognized command line option ‘-fstack-protector-strong’
gcc (GCC) 4.8.5

Our fork of nixpkgs is based on 16.09 and is here:
github.com/computecanada/nixpkgs

is there any commit I could pull to fix this ?

@globin
Copy link
Member

globin commented Feb 3, 2017

There is no optimal fix yet, workaround is still to export hardeningDisable=all in your shell while developing.

@mboisson
Copy link

mboisson commented Feb 3, 2017

Thanks.

@WesleyAC
Copy link

Just ran into this problem, and I agree that this hardening code should be deleted, or at the very least turned off be default. Overriding upstream compiler flags by default for everything is not a good idea.

@chrispickard
Copy link
Contributor

just lost two days to this bug, I'm really not sure why we are configuring gcc differently than everyone else outside of just ensuring it doesn't load from "system" directories. the error messages I got weren't even close to the issue (which is that fortify was turned on) so it took a long time to search random phrases and then eventually stumble on this issue. we have to do something about this. I believe I have even run into this bug before I just didn't realize it at the time.

@SohamG
Copy link
Contributor

SohamG commented Nov 22, 2023 via email

@bjornfor
Copy link
Contributor

AFAIR, nobody had a good solution to this issue until october this year: #18995 (comment). Someone "just" have to take that idea and apply it to the compiler wrapper?

@SohamG
Copy link
Contributor

SohamG commented Nov 22, 2023 via email

@jmarmstrong1207
Copy link
Contributor

Any updates @fpletz @cstrahan ?

@jmarmstrong1207
Copy link
Contributor

I'm not sure what the issue is though. These flags can be disabled and they are enabled by default not just in NixOS, but also in other distros such as Fedora and Ubuntu to increase security. Is there something I'm missing?

@zopsicle
Copy link

zopsicle commented Mar 8, 2024

Other distroes enable these flags when building packages, but not for general purpose compiler use (i.e. if the user is a programmer writing their own programs, not distro packages). Nixpkgs wraps the compilers to always set these flags, regardless of use case.

@vcunat
Copy link
Member

vcunat commented Mar 8, 2024

Yes, but Nix generally do not split these use cases. The normal way to do development is that you get nix-shell with everything set up as if doing a nix package build.

@zopsicle
Copy link

zopsicle commented Mar 8, 2024

I see. Then, is there a tool that can provide development environments with Nix packages, for when you’re not building a Nix package? I think "developing software that will exist outside the Nix ecosystem, using tools provided by Nix in an ephemeral environment" is a use case that should be officially supported, and is in fact something a lot of people currently abuse (apparently) nix-shell for.

Edit (2024-09-21)

I have been using nix-shell without stdenv for almost half a year now and must say that it has been very pleasant. 😃 The shell environment is precisely as specified in shell.nix, with no surprising magic behavior such as unwanted compiler flags, built-in CMake modules being shadowed, etc. The trick is to use builtins.derivation instead of stdenv.mkDerivation or mkShell for the shell, and set environment variables as desired from the setup script, like this:

derivation {
    name = "shell";
    builder = "${nixpkgs.coreutils}/bin/false";  # Only for use with nix-shell.
    system = nixpkgs.system;
    stdenv = nixpkgs.writeTextDir "setup" ''
        export PATH=${
            nixpkgs.lib.concatStringsSep ":" [
                "${nixpkgs.cmake}/bin"
                "${nixpkgs.coreutils}/bin"
                "${nixpkgs.gcc}/bin"  # Does not get unwanted flags!
                "${nixpkgs.pkg-config}/bin"
            ]
        }
        export PKG_CONFIG_PATH=${
            nixpkgs.lib.concatStringsSep ":" [
                "${nixpkgs.SDL2.dev}/lib/pkgconfig"
                "${nixpkgs.freetype.dev}/lib/pkgconfig"
            ]
        }
    '';
}

Adding a new dependency is a bit more work than "just add it to buildInputs" because you have to set CMake and/or pkg-config environment variables manually. But this is more than made up for by not having to debug problems caused by using stdenv for software development instead of Nixpkgs package authoring.

@jmarmstrong1207
Copy link
Contributor

I believe distrobox is a pretty good alternative and is what you're looking for

Jayman2000 added a commit to Jayman2000/sdl-log-example that referenced this issue May 12, 2024
This commit introduces a workaround for this problem:
<NixOS/nixpkgs#18995>. I decided to introduce
a workaround for that problem because I was getting a
fortify source–related warning.
jarmuszz added a commit to jarmuszz/scala-native that referenced this issue Jun 15, 2024
* Fixed `scala-native.nix`. Previous version had `-D_FORTIFY_SOURCE=2`
  implicitly set which issued lots of warnings when compiling native
  targets. See: NixOS/nixpkgs#18995
* Added support for setting up environment via flakes.
* Updated setup documentation and `docker/Dockerfile` to reflect above
  changes.
yipengsun added a commit to HamPepper/template-project-cpp that referenced this issue Oct 9, 2024
nixpkgs enables fortification by default, which adds `-O2` flag to gcc.
This breaks the debug build. For more info, see:

NixOS/nixpkgs#60919
NixOS/nixpkgs#18995

I also manually added `-O0` flag in debug flags in cmake.
This is, gcc would complain if I forgot to disable fortification.
srid added a commit to juspay/omnix that referenced this issue Oct 14, 2024
srid added a commit to juspay/omnix that referenced this issue Oct 15, 2024
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
hanno-becker pushed a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
mkannwischer added a commit to pq-code-package/mlkem-native that referenced this issue Jan 23, 2025
Before we would run into errors when compiling
with -O0 due to FORTIFY_SOURCE being set by the
compilers from the nix package manager.
Overwriting FORTIF_SOURCE=1 had no effect.
See NixOS/nixpkgs#18995.

We had worked around that by setting -O1 in a few
places in the CI.
The previous commit added hardeningDisable = [ "fortify" ];
to the respective compilers disabling that flag
 and it is hence no longer needed to avoid -O0 in CI.

This commit removes the -O1.

Signed-off-by: Matthias J. Kannwischer <matthias@kannwischer.eu>
midnightveil added a commit to midnightveil/nixpkgs that referenced this issue Feb 2, 2025
A very partial fix for [NixOS#18995]. This should make the relatively
common case of embedded targets Just Work™, since these systems rarely
are setup in the standard way.

I've often seen __stack__chk failures when building for embedded systems,
enabling this should make these architectures usable without needing
to do crimes in nix shells / build environments.

[NixOS#18995]: NixOS#18995
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 1.severity: blocker This is preventing another PR or issue from being completed
Projects
None yet
Development

Successfully merging a pull request may close this issue.