Skip to content

Commit

Permalink
stdenv/linux/default.nix: add gcc rebuild during bootstrap
Browse files Browse the repository at this point in the history 
before the change our rebuild chain was roughly the followin:

- `bootstrapTools` install
- `binutils` built by `bootstrapTools`
- `glibc` built by `bootstrapTools`
- `gcc` built by `bootstrapTools`

As a result `glibc` contained code generated by `bootstrapTools`'s
`gcc`. And (what worse) colied `libgcc_s.so.1` as is from it.

Such `libgcc_s.so.1` was not compatible with `nixpkgs` `gcc` at
least on `aarch64` where newer `libgcc` is expected to have new symbols.
As a result linking failed as:

    ld: /build/test.o: in function `foo(int)':
    test.cpp:(.text+0x2c): undefined reference to `__aarch64_ldadd4_acq_rel'
    collect2: error: ld returned 1 exit status

The change rejigs rebuild sequence as:

- `bootstrapTools` install
- `binutils` built by `bootstrapTools`
- `gcc` built by `bootstrapTools`
- `glibc` built by `gcc`
- `gcc` built by `gcc`

As a result `glibc` gets built by fresher `gcc` and embeds a copy
of `libgcc_s.so.1`.

To make it work as is I had to use a --sysroot=<invalid path> to evade
accidental pull of system headers embedded by `gcc`.

While at it decreased rebuild counts for `bash` and `xz`. These are
simple tools we can use as is built by `bootstrapTools` until we
build final stage.

List of packages with more than one build before the change:

    $ nix-store --query --graph $(nix-instantiate -A stdenv) |
        grep -P " -> " | awk '{print $3}' | sort -u | sed 's/"[0-9a-z]\{32\}-/"/g' |
        sort | uniq -c | sort -n | awk '$1 > 1'

      2 "autoconf-2.71.drv"
      2 "automake-1.16.5.drv"
      2 "bootstrap-stage1-stdenv-linux.drv"
      2 "bootstrap-stage2-stdenv-linux.drv"
      2 "bootstrap-stage4-stdenv-linux.drv"
      2 "bzip2-1.0.8.drv"
      2 "file-5.43.drv"
      2 "help2man-1.49.2.drv"
      2 "hook.drv"
      2 "libtool-2.4.7.drv"
      2 "patchelf-0.15.0.drv"
      2 "perl5.36.0-gettext-1.07.drv"
      3 "binutils-2.39.drv"
      3 "binutils-wrapper-2.39.drv"
      3 "expand-response-params.drv"
      3 "gettext-0.21.drv"
      3 "libxcrypt-4.4.33.drv"
      3 "perl-5.36.0.drv"
      3 "texinfo-6.8.drv"
      3 "zlib-1.2.13.drv"
      4 "bash-5.1-p16.drv"
      4 "xz-5.2.9.drv"

List of packages with more than one build after the change:

    $ nix-store --query --graph $(nix-instantiate -A stdenv) |
        grep -P " -> " | awk '{print $3}' | sort -u | sed 's/"[0-9a-z]\{32\}-/"/g' |
        sort | uniq -c | sort -n | awk '$1 > 1'

      2 "autoconf-2.71.drv"
      2 "automake-1.16.5.drv"
      2 "bash-5.2-p15.drv"
      2 "bootstrap-stage1-stdenv-linux.drv"
      2 "bootstrap-stage2-stdenv-linux.drv"
      2 "bootstrap-stage3-stdenv-linux.drv"
      2 "bootstrap-stage5-stdenv-linux.drv"
      2 "bzip2-1.0.8.drv"
      2 "file-5.43.drv"
      2 "gcc-11.3.0.drv"
      2 "gmp-with-cxx-6.2.1.drv"
      2 "help2man-1.49.2.drv"
      2 "hook.drv"
      2 "libtool-2.4.7.drv"
      2 "mpfr-4.1.1.drv"
      2 "patchelf-0.15.0.drv"
      2 "perl5.36.0-gettext-1.07.drv"
      2 "texinfo-6.8.drv"
      2 "which-2.21.drv"
      2 "xz-5.4.0.drv"
      3 "perl-5.36.0.drv"
      4 "binutils-2.39.drv"
      4 "binutils-wrapper-2.39.drv"
      4 "expand-response-params.drv"
      4 "gettext-0.21.drv"
      4 "libxcrypt-4.4.33.drv"
      4 "zlib-1.2.13.drv"
  • Loading branch information
@trofi
trofi committed Jan 4, 2023
1 parent 24309a2 commit 6428d11
Show file tree
Hide file tree
Showing 2 changed files with 85 additions and 46 deletions.
7 changes: 6 additions & 1 deletion pkgs/build-support/cc-wrapper/default.nix
View file
Expand Up @@ -331,12 +331,17 @@ stdenv.mkDerivation {
# compile, because it uses "#include_next <limits.h>" to find the
# limits.h file in ../includes-fixed. To remedy the problem,
# another -idirafter is necessary to add that directory again.
#
# We use --sysroot=/nix/dir/does/not/exist to drop embedded default
# path to glibc headers gcc was built against. Without it -idirafter
# only appends to the list and outdated glibc headers end up being
# used.
+ optionalString (libc != null) (''
touch "$out/nix-support/libc-cflags"
touch "$out/nix-support/libc-ldflags"
echo "-B${libc_lib}${libc.libdir or "/lib/"}" >> $out/nix-support/libc-crt1-cflags
'' + optionalString (!(cc.langD or false)) ''
echo "-idirafter ${libc_dev}${libc.incdir or "/include"}" >> $out/nix-support/libc-cflags
echo "--sysroot=/nix/dir/does/not/exist -idirafter ${libc_dev}${libc.incdir or "/include"}" >> $out/nix-support/libc-cflags
'' + optionalString (isGNU && (!(cc.langD or false))) ''
for dir in "${cc}"/lib/gcc/*/*/include-fixed; do
echo '-idirafter' ''${dir} >> $out/nix-support/libc-cflags
Expand Down
124 changes: 79 additions & 45 deletions pkgs/stdenv/linux/default.nix
View file
Expand Up @@ -9,14 +9,11 @@
# is used to build all other packages (including the bootstrapFiles).
#
# Goals of the bootstrap process:
# 1. final stdenv must not reference any of the bootstrap files.
# 2. final stdenv must not contain any of the bootstrap files
# (the only current violation is libgcc_s.so in glibc).
# 1. final stdenv must not reference any of the bootstrap packages.
# 2. final stdenv must not contain any of the bootstrap files.
# 3. final stdenv must not contain any of the files directly
# generated by the bootstrap code generators (assembler, linker,
# compiler). The only current violations are: libgcc_s.so in glibc,
# the lib{mpfr,mpc,gmp,isl} which are statically linked
# into the final gcc).
# compiler).
#
# These goals ensure that final packages and final stdenv are built
# exclusively using nixpkgs package definitions and don't depend
Expand Down Expand Up @@ -109,12 +106,13 @@ let
'';


# The bootstrap process proceeds in several steps.


# Create a standard environment by downloading pre-built binaries of
# coreutils, GCC, etc.

# The bootstrap process proceeds in several steps. Our high-level plan
# is the following:
#
# - build binutils with bootstraTools
# - build gcc with bootstraTools
# - build glibc with bootstraTools (embed libgcc_s.so.1)
# - build gcc with nixpkgs

# Download and unpack the bootstrap tools (coreutils, GCC, Glibc, ...).
bootstrapTools = import (if localSystem.libc == "musl" then ./bootstrap-tools-musl else ./bootstrap-tools) {
Expand Down Expand Up @@ -284,21 +282,63 @@ in
};
})


# 2nd stdenv that contains our own rebuilt binutils and is used for
# compiling our own Glibc.
# compiling our own gcc.
#
# resulting stage2 stdenv:
# - coreutils, glibc, gcc: from bootstrapFiles
# - binutils: from nixpkgs, built by bootstrapFiles toolchain
# - coreutils, glibc: from bootstrapFiles
# - binutils, gcc: from nixpkgs, built by bootstrapFiles toolchain
(prevStage: stageFun prevStage {
name = "bootstrap-stage2";

overrides = self: super: {
inherit (prevStage)
ccWrapperStdenv
gcc-unwrapped coreutils gnugrep
perl gnum4 bison;
bash bison gnum4 coreutils gnugrep
patchelf perl texinfo xz;

${localSystem.libc} = getLibc prevStage;

gcc-unwrapped =
let makeStaticLibrariesAndMark = pkg:
lib.makeOverridable (pkg.override { stdenv = self.makeStaticLibraries self.stdenv; })
.overrideAttrs (a: { pname = "${a.pname}-stage2"; });
in super.gcc-unwrapped.override {
# Link GCC statically against GMP etc. This makes sense because
# these builds of the libraries are only used by GCC, so it
# reduces the size of the stdenv closure.
gmp = makeStaticLibrariesAndMark super.gmp;
mpfr = makeStaticLibrariesAndMark super.mpfr;
libmpc = makeStaticLibrariesAndMark super.libmpc;
isl = makeStaticLibrariesAndMark super.isl_0_20;
# Use a deterministically built compiler
# see https://github.com/NixOS/nixpkgs/issues/108475 for context
reproducibleBuild = true;
profiledCompiler = false;
};
};

# `libtool` comes with obsolete config.sub/config.guess that don't recognize Risc-V.
extraNativeBuildInputs =
lib.optional (localSystem.isRiscV) prevStage.updateAutotoolsGnuConfigScriptsHook;
})

# 3rd stdenv that contains our own rebuilt binutils and gcc and is
# used for compiling our own Glibc.
#
# resulting stage3 stdenv:
# - coreutils: from bootstrapFiles
# - glibc, binutils, gcc: from nixpkgs, built by bootstrapFiles toolchain
(prevStage: stageFun prevStage {
name = "bootstrap-stage3";

overrides = self: super: {
inherit (prevStage)
ccWrapperStdenv
gcc-unwrapped
bash bison coreutils gnum4 gnugrep
patchelf perl;

dejagnu = super.dejagnu.overrideAttrs (a: { doCheck = false; } );

# We need libidn2 and its dependency libunistring as glibc dependency.
Expand Down Expand Up @@ -356,21 +396,24 @@ in
})


# Construct a third stdenv identical to the 2nd, except that this
# Construct a fourth stdenv identical to the 3rd, except that this
# one uses the rebuilt Glibc from stage2. It still uses the recent
# binutils and rest of the bootstrap tools, including GCC.
#
# resulting stage3 stdenv:
# resulting stage4 stdenv:
# - coreutils, gcc: from bootstrapFiles
# - glibc, binutils: from nixpkgs, built by bootstrapFiles toolchain
# - binutils, gcc: from nixpkgs, built by bootstrapFiles toolchain
# - glibc: from nixpkgs, built by nixpkgs toolchain
(prevStage: stageFun prevStage {
name = "bootstrap-stage3";
name = "bootstrap-stage4";

overrides = self: super: rec {
inherit (prevStage)
ccWrapperStdenv
binutils coreutils gnugrep
perl patchelf linuxHeaders gnum4 bison libidn2 libunistring;
bison binutils coreutils gnugrep gnum4
patchelf perl texinfo xz
linuxHeaders libidn2 libunistring;

${localSystem.libc} = getLibc prevStage;
gcc-unwrapped =
let makeStaticLibrariesAndMark = pkg:
Expand All @@ -397,28 +440,27 @@ in
})


# Construct a fourth stdenv that uses the new GCC. But coreutils is
# Construct a fifth stdenv that uses the new GCC. But coreutils is
# still from the bootstrap tools.
#
# resulting stage4 stdenv:
# resulting stage5 stdenv:
# - coreutils: from bootstrapFiles
# - glibc, binutils: from nixpkgs, built by bootstrapFiles toolchain
# - gcc: from nixpkgs, built by bootstrapFiles toolchain. Can assume
# it has almost no code from bootstrapTools as gcc bootstraps
# internally. The only exceptions are crt files from glibc
# built by bootstrapTools used to link executables and libraries,
# and the bootstrapTools-built, statically-linked
# lib{mpfr,mpc,gmp,isl}.a which are linked into the final gcc
# (see commit cfde88976ba4cddd01b1bb28b40afd12ea93a11d).
# - binutils: from nixpkgs, built by bootstrapFiles toolchain
# - glibc, gcc: from nixpkgs, built by nixpkgs toolchain
(prevStage: stageFun prevStage {
name = "bootstrap-stage4";
name = "bootstrap-stage5";

overrides = self: super: {
# Zlib has to be inherited and not rebuilt in this stage,
# because gcc (since JAR support) already depends on zlib, and
# then if we already have a zlib we want to use that for the
# other purposes (binutils and top-level pkgs) too.
inherit (prevStage) gettext gnum4 bison perl texinfo zlib linuxHeaders libidn2 libunistring;
inherit (prevStage)
bash gettext gnum4 bison
perl texinfo which
# xz - rebuild to pull in rebuilt version into final stdenv
zlib linuxHeaders libidn2 libunistring;

${localSystem.libc} = getLibc prevStage;
binutils = super.binutils.override {
# Don't use stdenv's shell but our own
Expand Down Expand Up @@ -468,15 +510,7 @@ in
# binutils built.
#
# resulting stage5 (final) stdenv:
# - coreutils, binutils: from nixpkgs, built by nixpkgs toolchain
# - glibc: from nixpkgs, built by bootstrapFiles toolchain
# - gcc: from nixpkgs, built by bootstrapFiles toolchain. Can assume
# it has almost no code from bootstrapTools as gcc bootstraps
# internally. The only exceptions are crt files from glibc
# built by bootstrapTools used to link executables and libraries,
# and the bootstrapTools-built, statically-linked
# lib{mpfr,mpc,gmp,isl}.a which are linked into the final gcc
# (see commit cfde88976ba4cddd01b1bb28b40afd12ea93a11d).
# - coreutils, binutils, glibc, gcc: from nixpkgs, built by nixpkgs toolchain
(prevStage: {
inherit config overlays;
stdenv = import ../generic rec {
Expand Down

0 comments on commit 6428d11

Please sign in to comment.