Skip to content

Building gcc with glibc together in 10 easy steps

Jump to bottom Edit New page
Harmen Stoppels edited this page Sep 18, 2023 · 2 revisions

With https://github.com/spack/spack/pull/40013 it should be possible to build new gcc with old glibc.

Here are my current notes. This assumes you have an empty directory $work.

  1. Create an environment in $work with the relevant GCC and glibc

    spack:
  specs:
  - glibc@2.5
  - gcc@13
  view: view
  concretizer:
    unify: true

  2. Install all dependencies, but not gcc and glibc -- they're built together by hand

  3. Run spack -e . patch gcc glibc to download, stage and patch

  4. Move the sources

    $ mv `spack -e . location gcc` gcc-src
$ mv `spack -e . location glibc` glibc-src

  5. Install the glibc headers:

    $ export linux_headers="$(spack -e . location -i linux-headers)/include"
$ spack -e . build-env glibc -- bash
$ mkdir glibc-hdrs && cd glibc-hdrs
$ CXX=/nonsense ../glibc-src/configure \
    --prefix=$work/glibc-install \
    --with-headers=$linux_headers \
    --enable-kernel=6.2.0
$ make install-headers cross-compiling=yes -j $(nproc)
$ exit # the build env shell

    So here we configure glibc with the "wrong" compiler, but it's all we have. The cross-compiling=yes is necessary for older glibc that ships sunrpc; without it it attempts to compile/link smth, which fails.

  6. Manually create dummy versions of missing headers (not all bootstrap headers are included in install):

    $ touch $work/glibc-install/include/gnu/stubs.h
$ touch $work/glibc-install/include/bits/stdio_lim.h
$ touch $work/glibc-install/include/bits/syscall.h

    TODO: are these all?

  7. Then configure GCC once and only once

    TODO: should set LD_FOR_TARGET and friends, otherwise GCC stores the compiler wrapper, and it gets used when building glibc, which is problematic cause we can't have an rpath for ld.so.

    $ spack -e . build-env gcc -- bash
$ mkdir gcc-obj && cd gcc-obj
$ LDFLAGS_FOR_TARGET="-L$work/glibc-install/lib -Wl,-rpath,$work/glibc-install/lib,-dynamic-linker,$work/glibc-install/lib/ld-linux-x86-64.so.2" CFLAGS_FOR_TARGET="-isystem $linux_headers" CXXFLAGS_FOR_TARGET="-isystem $linux_headers" ../gcc-src/configure \
    --prefix=$work/gcc-install \
    --disable-bootstrap \
    --enable-languages=c,c++ \
    --disable-multilib \
    --with-native-system-header-dir=$work/glibc-install/include \
    --with-gmp=$work/view

    Note: since we have glibc and linux headers in separate packages, we need that extra -isystem flag. For some reason, GCC does not save the flag CPPFLAGS_FOR_TARGET, only [C|CXX]FLAGS_FOR_TARGET, so we use that.

  8. First build libgcc.a only. Since we haven't configured gcc with --disable-shared, we need to override enable_shared=no, otherwise libgcc_eh.a and libgcc_s.so build too:

    $ make all-target-libgcc enable_shared=no -j $(nproc)
$ exit # the build env

  9. The glibc we're using is patched on older version so it doesn't need libgcc_eh.a / libgcc_s.so, so now we should be able to configure it with just-built xgcc.

    Some configure tests need to be disabled because they require executables to be created or run (but xgcc cannot yet)

    A bunch of CFLAGS are needed for older versions, but Spack's build env should already do that

    Override CXX to garbage since we don't have a g++ compiler, and if glibc detects a system g++, it will start building tests (which fails to link). Also, we should avoid hitting Spack's linker wrapper here, so remove the wrapper dir from PATH (!). If not, you'll end up with ld.so built with rpaths, which causes startup errors of the dynamic linker. I also had to modify the wrappers in $work/gcc-obj/gcc/collect-ld etc that still referenced the spack linker wrapper, and replace it with /usr/bin/ld.

    $ spack -e . build-env glibc -- bash
$ mkdir glibc-obj && cd glibc-obj
$ export CC="$work/gcc-obj/gcc/xgcc -B $work/gcc-obj/gcc/" CXX=/no/no/no 
$ ../glibc-src/configure \
    --with-headers=$linux_headers \
    --prefix=$work/glibc-install \
    --enable-kernel=6.2.0 \
    libc_cv_forced_unwind=yes \
    libc_cv_c_cleanup=yes \
    ac_cv_sizeof_long_double=8
$ make -j $(nproc)
$ make install
$ exit

  10. Return to GCC and install the rest, now with TFLAGS so glibc crt*.o files are picked up for building shared libs (the other flags were already set when we configured gcc):

$ spack -e . build-env gcc -- bash
$ cd gcc-obj
$ make TFLAGS="-B$work/glibc-install/lib" -j $(nproc)
$ make install
$ exit

Now you should have GCC in $work/gcc-install with runtime libs linked against glibc in $work/glibc-install.

Caveat: Old glibc ldd gets the lib dir wrong (on ubuntu at least): it has lib64/ld.so, but it lives in in lib/.

To use the compiler, a spec file is useful to deal with all custom locations; I've adapted it from gcc -dumpspecs; some rules can be extended, but startfile and endfile cannot easily. My glibc.spec looks as follows (with $work and $linux_headers the actual absolute paths) and can be used with gcc -specs=glibc.spec ...:

*cpp:
+ -isystem $linux_headers

*link_libgcc:
+ -rpath $work/gcc-install/lib64/

*link:
+ -dynamic-linker $work/glibc-install/lib/ld-2.5.so -L $work/glibc-install/lib/ -rpath $work/glibc-install/lib/

*endfile:
%{!mandroid|tno-android-ld:%{mdaz-ftz:crtfastmath.o%s;Ofast|ffast-math|funsafe-math-optimizations:%{!shared:%{!mno-daz-ftz:crtfastmath.o%s}}}    %{mpc32:crtprec32.o%s}    %{mpc64:crtprec64.o%s}    %{mpc80:crtprec80.o%s} %{!static:%{fvtable-verify=none:%s;      fvtable-verify=preinit:vtv_end_preinit.o%s;      fvtable-verify=std:vtv_end.o%s}}    %{static:crtend.o%s;      shared|static-pie|pie:crtendS.o%s;      :crtend.o%s} $work/glibc-install/lib/crtn.o%s ;:%{mdaz-ftz:crtfastmath.o%s;Ofast|ffast-math|funsafe-math-optimizations:%{!shared:%{!mno-daz-ftz:crtfastmath.o%s}}}    %{mpc32:crtprec32.o%s}    %{mpc64:crtprec64.o%s}    %{mpc80:crtprec80.o%s} %{shared: crtend_so%O%s;: crtend_android%O%s}}

*startfile:
%{!mandroid|tno-android-ld:%{shared:;      pg|p|profile:%{static-pie:grcrt1.o%s;:$work/glibc-install/lib/gcrt1.o%s};      static:$work/glibc-install/lib/crt1.o%s;      static-pie:rcrt1.o%s;      pie:$work/glibc-install/lib/Scrt1.o%s;      :$work/glibc-install/lib/crt1.o%s} $work/glibc-install/lib/crti.o%s    %{static:crtbeginT.o%s;      shared|static-pie|pie:crtbeginS.o%s;      :crtbegin.o%s}    %{fvtable-verify=none:%s;      fvtable-verify=preinit:vtv_start_preinit.o%s;      fvtable-verify=std:vtv_start.o%s} ;:%{shared: crtbegin_so%O%s;:  %{static: crtbegin_static%O%s;: crtbegin_dynamic%O%s}}}
Add a custom footer
Clone this wiki locally