stdenv: let aarch64 use gcc ≥10 #108305

FRidh · 2021-01-03T10:26:39Z

Describe the bug
Nixpkgs upgraded to gcc 10 in #89793. Unfortunately, there were issues with aarch64 so aarch64 still uses gcc 9.

stale · 2021-11-09T19:21:31Z

I marked this as stale due to inactivity. → More info

ilya-fedin · 2022-07-22T20:24:47Z

Is there any progress on this issue? I maintain a package that has ICE with gcc 9 and has #108111 with gcc 10 on aarch64, so it was marked as broken on aarch64 in #173671 (and I have completely no idea how to un-break it as it seems all nixpkgs compilers just fail on it due to those issues).

ajs124 · 2022-07-22T20:29:25Z

See #167726 (comment)

dotlambda · 2022-07-24T23:10:48Z

In #153329 adding gcc11 to nativeBuildInputs did the trick.

ilya-fedin · 2022-07-24T23:13:43Z

🤔 changing stdenv to gcc11Stdenv doesn't work?

dotlambda · 2022-07-25T03:03:22Z

🤔 changing stdenv to gcc11Stdenv doesn't work?

True. That's a cleaner solution if it does work in your case.

K900 · 2022-10-12T09:24:53Z

Another blocker: #191357

K900 · 2023-01-15T09:46:33Z

Another victim: #210859

@Ericson2314

#### Immediate Benefits - Allow `gcc11` on `aarch64` - No more copying `libgcc_s` out of the bootstrap-files or other derivations - No more [static `lib{mpfr,mpc,gmp,isl}.a` hack](https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380) - *Zero* additional `gcc` builds (stage1+stage2+stageCompare) - The `gcc` derivation builds `gcc` once instead of three times. - The libraries that are linked into the final `pkgs.gcc` (`mpfr`, `mpc`, `gmp`, `isl`, `glibc`) are built by `stdenv.__bootPkgs.gcc` rather than by the `bootstrapFiles`. No more Frankenstein compiler! - stageCompare runs **concurrently** with (not in series with) with `stdenv`'s dependees. - Many other `stdenv` hacks eliminated. - `gcc` and `clang` share the same codepath for more of `cc-wrapper`. - Makes the cross and native codepaths much more similar -- another step towards "cross by default". Note that *all* the changes in this PR are controlled by flags; no old codepaths need to be removed until/if we're completely certain that this is the right way to go. #### Future Benefits - This should allow using a [foreign] `bootstrap-files` so long as `hostPlatform.canExecute bootstrapFiles`. - There will be an "avalanche of simplification" when we set `enableGccExternalBootstrap=true` and run dead code elimination. It's really quite a huge amount of code that goes away. Native-gcc has its own special codepath in so many places, while cross-gcc and clang work the same way (and are much simpler). - This should allow each of the libraries that ship with `gcc` (`lib{backtrace,atomic,cc1,decnumber,ffi,gomp,iberty,offloadatomic,quadmath,sanitizer,ssp,stdc++-v3,vtv}`) to be built in separate (one-liner) derivations which `inherit src;` from `gcc`. - Building `libstdc++-v3` in a separate derivation will eliminate a lot of accidental-reference-to-the-`bootstrapFiles` landmines. #### Incorporates - NixOS#209054 - NixOS#210004 - NixOS#36948 (unreverted) - NixOS#210325 - NixOS#210118 - NixOS#210132 - NixOS#210109 #### Closes - Closes NixOS#208412 - Closes NixOS#108111 - Closes NixOS#108305 - Closes NixOS#201254 #### Build history - First successful builds (stage1/stage2): - powerpc64le-linux at 9c7e9ef - x86_64-linux at 9c7e9ef - aarch64-linux at 4d5bc7d - First successful comparisons (stageCompare): - at 81949cf - [aarch64-linux][aarch64-compare-ofborg] - [x86\_64-linux][amd64-compare-ofborg] #### Credits This project was made possible by three important insights, none of which were mine: 1. @Ericson2314 was the first to advocate for this change, and probably the first to appreciate its advantages. External bootstrap is "cross by default". 2. @trofi has figured out a lot about how to get gcc to not mix up the copy of `libstdc++` that it depends on with the copy that it builds. Now that gcc is written in C++, it depends on `libstdc++`, builds a copy of `libstdc++`, and builds auxiliary products (like `libplugin`) which depend on `libstdc++`. @trofi developed two important techniques for keeping this straight: the use of a [nonexistent sysroot] and moving the `bootstrapFiles`' `libstdc++` into a [versioned directory]. Without these two discoveries, external bootstrap would be impossible, because the final gcc would still have references to the `bootstrapFiles`. 3. Using the undocumented variable [`user-defined-trusted-dirs`] when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it uses a completely different and totally special set of rules for finding `libgcc_s.so`. This trick is the only way we can put `libgcc_s.so` in its own separate outpath without creating circular dependencies or dependencies on the bootstrapFiles. I would never have guessed to use this (or that it existed!) if it were not for a [comment in guix] which @Mic92 [mentioned]. My own role in this PR was basically: being available to go on a coding binge at an opportune moment, so we wouldn't waste a [crisis]. [aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938 [amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857 [nonexistent sysroot]: NixOS#210004 [versioned directory]: NixOS#209054 [`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html [comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253 [mentioned]: NixOS#210112 (comment) [crisis]: NixOS#108305 [foreign]: NixOS#170857 (comment)

roberth · 2023-02-01T19:19:18Z

Another victim:

Proposal: Use C++20 nix#7203

roberth · 2023-02-01T19:36:59Z

Potential solution:

stdenv: Nix-driven bootstrap of gcc #209870

(If I may summarize what can be inferred from the issue page)

ghost · 2023-02-12T18:44:56Z

FWIW the commit message and leading comment for #209870 both contain this text:

- Closes #108305

but for some reason github isn't processing it.

ilya-fedin · 2023-02-12T18:46:58Z

but for some reason github isn't processing it.

K900 · 2023-02-12T18:47:38Z

Github is having a pretty bad day it seems, it'll probably catch up in a bit.

ghost · 2023-02-13T20:07:01Z

but for some reason github isn't processing it.

Specifically, this PR doesn't show a "merging #209870 may close this bug" or list this as a "linked issue" of that PR. It should say "Development Successfully merging this pull request may close these issues: #209870"

For an example of what it should look like, check out e.g. number 216163 (no hyperlink to avoid noise in that PR).

K900 · 2023-02-14T08:09:30Z

Note to self: 6d6a980 will need to be reverted when any of the solutions land.

dotlambda · 2023-03-16T22:12:54Z

@wegank Can you link the PR in which this was fixed?

@Ericson2314

#### Summary By default, when you type `make`, GCC will compile itself three times. This PR inhibits that behavior by configuring GCC with `--disable-bootstrap`, and reimplements the triple-rebuild using Nix rather than `make`/`sh`. #### Immediate Benefits - Allow `gcc11` and `gcc12` on `aarch64` (without needing new `bootstrapFiles`) - Faster stdenv rebuilds: the third compilation of gcc (i.e. stageCompare) is no longer a `drvInput` of the final stdenv. This allows Nix to build stageCompare in parallel with the rest of nixpkgs instead of in series. - No more copying `libgcc_s` out of the bootstrap-files or other derivations - No more Frankenstein compiler: the final gcc and the libraries it links against (mpfr, mpc, isl, glibc) are all built by the same compiler (xgcc) instead of a mixture of the bootstrapFiles' compiler and xgcc. - No more [static lib{mpfr,mpc,gmp,isl}.a hack] - Many other small `stdenv` hacks eliminated - `gcc` and `clang` share the same codepath for more of `cc-wrapper`. #### Future Benefits - This should allow using a [foreign] `bootstrap-files` so long as `hostPlatform.canExecute bootstrapFiles`. - This should allow each of the libraries that ship with `gcc` (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty, offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be built in separate (one-liner) derivations which `inherit src;` from `gcc`, much like NixOS#132343 #### Incorporates - NixOS#210004 - NixOS#36948 (unreverted) - NixOS#210325 - NixOS#210118 - NixOS#210132 - NixOS#210109 - NixOS#213909 - NixOS#216136 - NixOS#216237 - NixOS#210019 - NixOS#216232 - NixOS#216016 - NixOS#217977 - NixOS#217995 #### Closes - Closes NixOS#108305 - Closes NixOS#108111 - Closes NixOS#201254 - Closes NixOS#208412 #### Credits This project was made possible by three important insights, none of which were mine: 1. @Ericson2314 was the first to advocate for this change, and probably the first to appreciate its advantages. Nix-driven (external) bootstrap is "cross by default". 2. @trofi has figured out a lot about how to get gcc to not mix up the copy of `libstdc++` that it depends on with the copy that it builds, by moving the `bootstrapFiles`' `libstdc++` into a [versioned directory]. This allows a Nix-driven bootstrap of gcc without the final gcc would still having references to the `bootstrapFiles`. 3. Using the undocumented variable [`user-defined-trusted-dirs`] when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it uses a completely different and totally special set of rules for finding `libgcc_s.so`. This trick is the only way we can put `libgcc_s.so` in its own separate outpath without creating circular dependencies or dependencies on the bootstrapFiles. I would never have guessed to use this (or that it existed!) if it were not for a [comment in guix] which @Mic92 [mentioned]. My own role in this PR was basically: being available to go on a coding binge at an opportune moment, so we wouldn't waste a [crisis]. [aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938 [amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857 [nonexistent sysroot]: NixOS#210004 [versioned directory]: NixOS#209054 [`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html [comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253 [mentioned]: NixOS#210112 (comment) [crisis]: NixOS#108305 [foreign]: NixOS#170857 (comment) [static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380

FRidh added the 0.kind: bug label Jan 3, 2021

FRidh added this to the 21.03 milestone Jan 3, 2021

FRidh added the 1.severity: blocker label Jan 4, 2021

This was referenced Jan 4, 2021

stdenv: use gcc9 for aarch64 #108200

Merged

aarch64 build failures with gcc 10 and binutils 2.34 #108111

Closed

FRidh changed the title ~~let aarch64 use gcc 10~~ stdenv: let aarch64 use gcc 10 Jan 4, 2021

FRidh added the 6.topic: stdenv Standard environment label Jan 4, 2021

stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Nov 9, 2021

sternenseemann removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Nov 30, 2021

ajs124 mentioned this issue Jan 3, 2022

tdesktop: 3.1.11 -> 3.4.3 #152702

Merged

13 tasks

ajs124 mentioned this issue Apr 7, 2022

gcc: upgrade aarch64-linux default from 9 to 11 #167726

Closed

13 tasks

ajs124 mentioned this issue Apr 22, 2022

proxysql: fix build #169832

Merged

13 tasks

dotlambda mentioned this issue May 13, 2022

libsForQt5.plasmaMobileGear: 21.08 -> 22.04 #153329

Merged

13 tasks

stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 19, 2022

Artturin removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 3, 2022

Artturin changed the title ~~stdenv: let aarch64 use gcc 10~~ stdenv: let aarch64 use gcc ≥10 Aug 18, 2022

Artturin mentioned this issue Sep 2, 2022

vte: fix cross #183278

Closed

13 tasks

zzywysm mentioned this issue Nov 28, 2022

build the kernel with a newer gcc tpwrules/nixos-apple-silicon#11

Closed

Artturin mentioned this issue Dec 12, 2022

llvmPackages_14: Fix build on aarch64-linux #201485

Merged

13 tasks

Artturin modified the milestones: 21.05, 23.05 Dec 31, 2022

This was referenced Jan 7, 2023

aarch64: enable rebootstrap, upgrade to gcc11 #209462

Closed

stdenv: Nix-driven bootstrap of gcc #209870

Merged

roberth mentioned this issue Feb 1, 2023

Proposal: Use C++20 NixOS/nix#7203

Merged

ilya-fedin mentioned this issue Mar 16, 2023

kotatogram-desktop: unbreak on aarch64-linux #221485

Merged

12 tasks

wegank closed this as completed Mar 16, 2023

wegank mentioned this issue Mar 16, 2023

stdenv: aarch64-linux: gcc9 -> gcc12 #207135

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stdenv: let aarch64 use gcc ≥10 #108305

stdenv: let aarch64 use gcc ≥10 #108305

FRidh commented Jan 3, 2021 •

edited by dotlambda

stale bot commented Nov 9, 2021

ilya-fedin commented Jul 22, 2022 •

edited

ajs124 commented Jul 22, 2022

dotlambda commented Jul 24, 2022

ilya-fedin commented Jul 24, 2022

dotlambda commented Jul 25, 2022

K900 commented Oct 12, 2022

K900 commented Jan 15, 2023

roberth commented Feb 1, 2023

roberth commented Feb 1, 2023

ghost commented Feb 12, 2023

ilya-fedin commented Feb 12, 2023 •

edited

K900 commented Feb 12, 2023

ghost commented Feb 13, 2023

K900 commented Feb 14, 2023

dotlambda commented Mar 16, 2023

stdenv: let aarch64 use gcc ≥10 #108305

stdenv: let aarch64 use gcc ≥10 #108305

Comments

FRidh commented Jan 3, 2021 • edited by dotlambda

stale bot commented Nov 9, 2021

ilya-fedin commented Jul 22, 2022 • edited

ajs124 commented Jul 22, 2022

dotlambda commented Jul 24, 2022

ilya-fedin commented Jul 24, 2022

dotlambda commented Jul 25, 2022

K900 commented Oct 12, 2022

K900 commented Jan 15, 2023

roberth commented Feb 1, 2023

roberth commented Feb 1, 2023

ghost commented Feb 12, 2023

ilya-fedin commented Feb 12, 2023 • edited

K900 commented Feb 12, 2023

ghost commented Feb 13, 2023

K900 commented Feb 14, 2023

dotlambda commented Mar 16, 2023

FRidh commented Jan 3, 2021 •

edited by dotlambda

ilya-fedin commented Jul 22, 2022 •

edited

ilya-fedin commented Feb 12, 2023 •

edited