This repository has been archived by the owner. It is now read-only.
scripts to build a fresh Archlinux32 using crosstool-ng (Moved to https://git.archlinux32.org/archlinux32/bootstrap32)
Switch branches/tags
Nothing to show
Clone or download
Andreas Baumann
Latest commit 20cb273 Apr 20, 2018
Permalink
Failed to load latest commit information.
.Attic built iputils and gnupg for stage 3 Mar 4, 2018
i486-stage1 building up to meson Mar 10, 2018
i486-stage2 building git, some fixes with login shells in ssh Mar 23, 2018
i486-stage3 building up to guile Mar 25, 2018
i486-stage4 added namcap and dependencies Apr 20, 2018
README added some missing packages in stage4 Apr 2, 2018
TODOS added some missing packages in stage4 Apr 2, 2018
build_cross.sh fixed all shellcheck bugs Feb 17, 2018
build_stage1.sh fixed all shellcheck bugs Feb 17, 2018
build_stage1_package.sh cleanup and plan for stage4 Mar 15, 2018
build_stage2.sh finished stage2 (but for syslinux) Feb 24, 2018
build_stage2_package.sh cleanup and plan for stage4 Mar 15, 2018
build_stage3.sh finished up to linux kernel Mar 14, 2018
build_stage3_package.sh cleanup and plan for stage4 Mar 15, 2018
build_stage4.sh added namcap and dependencies Apr 20, 2018
build_stage4_package.sh building up to guile Mar 25, 2018
compute_dependencies.sh some work on the dependency calculator for stage 2, stage 3 Feb 18, 2018
create_ca-certificates-utils_shim.sh fixed all shellcheck bugs Feb 17, 2018
create_gcc-lib_shim.sh fixed all shellcheck bugs Feb 17, 2018
create_glibc_shim.sh fixed all shellcheck bugs Feb 17, 2018
create_hdd.sh building git, some fixes with login shells in ssh Mar 23, 2018
create_stage1_cdrom.sh added archiso32-git for CDROM building Apr 6, 2018
create_stage4_cdrom.sh building standard ISO for systems with more than 256 MB of RAM Apr 7, 2018
ct-ng.config initial checkin, building crosstool-ng Jan 26, 2018
default.conf added archiso32-git for CDROM building Apr 6, 2018
install_host.sh building quite some more stage3 packages Mar 3, 2018
prepare_stage1_repo.sh fixed all shellcheck bugs Feb 17, 2018
prepare_stage2_repo.sh building up to libarchive Feb 23, 2018
prepare_stage3_repo.sh cleanup and plan for stage4 Mar 15, 2018
prepare_stage4_repo.sh cleanup and plan for stage4 Mar 15, 2018

README

Mission
-------

Build a crosstool-ng toolchain for i486.

Use makepkg with patched PKGBUILDs for cross-compilation.

We cannot run tests this way, as we don't assume the i486
is running on x64_64 (which it does, but for the sake
of the general usage of the approach we MUST assume it
doesn't!).

Packages built for the chroot must be also accessible to
the cross-compiler (for header files, tools and libraries).
For this we unpack all generated chroot packages in the sysroot
of the toolchain.

x-tools and sysroot: the cross-compiler (cross-ng) and
the installed packages (using bsdtar) in the sysroot
of the toolchain.

i486-root destination root, being the boot image of the
temporary system for i486.

The goal is to get to a self-contained system with as
few packages as possible which allows a basic i486 to
be built. This requires at least some development tools,
pacman and some base system to be cross-compiled.

We also want basic network connetivity, so we can use
the i486 VMs or real i486 as build slaves. Also, having
a minimal bootable ISO for the i486 system (or even a
set of floppies or PXE boot via a boot floppy would
be very useful). We cannot have kernel modules in a 
ramdisk or systemd for this basic system because they
interfere heavily with memory constains on a 486! So,
we build up to an openssh server.

On top, the full Archlinux i486 universe can then be
built (either with a i486 chroot or via a i486 master/
distcc slaves setup or a distributed i486 cluster setup).

There is another caveat: we are patching the PKGBUILDs
to do proper cross-compilation (--with-host, --with-target).
Still we try to stay close to the original PKGBUILDs.

References
----------

https://archlinuxarm.org/wiki/Distcc_Cross-Compiling
https://wiki.archlinux.org/index.php/Cross-compiling_tools_package_guidelines
http://mgalgs.github.io/2011/12/08/creating-arch-linux-packages-by-hand.html
http://clfs.org/view/svn/ppc/chroot/before-chroot.html
http://www.linuxfromscratch.org/clfs/
https://www.dotslashlinux.com/2017/04/29/booting-the-linux-kernel-without-an-initrd-initramfs/
https://rubenerd.com/sata-on-qemu/
https://archlinuxarm.org/forum/viewtopic.php?f=57&t=6163
https://dustymabe.com/2012/12/15/mounting-a-partition-within-a-disk-image/
http://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/
http://wiki.osdev.org/GCC_Cross-Compiler
http://www.linuxfromscratch.org/lfs/view/6.2/chapter05/adjusting.html
https://wiki.debian.org/FakeRoot
https://archlinuxarm.org/forum/viewtopic.php?f=57&t=6163
https://www.phenix.bnl.gov/~purschke/RescueCD/
https://github.com/ivandavidov/minimal-linux-script
https://askubuntu.com/questions/109413/how-do-i-use-overlayfs
https://how-to-build-for-arm.wikispaces.com/
https://arsv.github.io/perl-cross/
http://www.ibb.net/~anne/keyboard.html

Recipe
------

##############
# PREPARATIONS
##############

# Prepare the host. We use Archlinux 64-bit as we don't fully trust the
# 32-bit toolchain in Archlinux32 (yet). And a 64-bit based toolchain is
# more usable anyway nowadays.

# Archlinux with base and base-devel (2018.01.01-x86_64.iso).
# OpenSSH and editor (joe), grub.

# Install necessary tools

./install_host.sh

# Prepare the cross-compiler for the destination platform, in our
# case i486.
#
# configuration of crosstools-ng:
#
# Target architecture i486
# Bitness 32-bit
# Architecture level i486
# Emit assembly for CPU i486
# Toolchain bug URL: https://bugs.archlinux32.org
# Type: Cross
# Build System: empty (was: x86_64-pc-linux-gnu)
# No NLS
# Target OS linux
# version of linux 4.13.9
# binutils 2.29.1
# select ld, gold as linkers
# C-library: glibc
# gcc 7.2.0
# no libmpx
# Don't forget to enable C++!

ct-ng menuconfig
cp .config ct-ng.config

# build the toolchain, results in a 486 toolchain in /home/cross/x-tools

./build_cross.sh

# prepare repo where we build stage 1 with the cross-compiler

su cross ./prepare_stage1_repo.sh

#########
# STAGE 1
#########

# Build stage1 in $STAGE1_BUILD with the cross-compiler and modified
# PKGBUILDs and patches into $STAGE1_CHROOT.

# If necessary to fullfill the dependencies of makepkg, we create phantom
# packages by hand without makepkg with files from the toolchain's sysroot.

# So, we build gcc-libs for compiler libraries (C and C++) and the glibc
# we will NOT build them using the cross-compiler as packages but later
# at the end of phase 1. This ensures glibc and gcc-libs are around as
# dependency for makepkg all the time, if not the original one, so close
# ones fitting to the crosstoolchain.

# In this stage we are not allowed to have loops when building, the build
# tree must be a real tree.

# The goal is to get a booting system of the target architecture (for instance
# to be installed on real hardware for a build cluster).

su cross ./create_gcc-lib_shim.sh

su cross ./create_glibc_shim.sh

# take pre-computed cert stores from an existing machine as the
# dependencies to build the package are just too many

su cross ./create_ca-certificates-utils_shim.sh

# Build stage 1 and install it into a chroot and into sysroot

./build_stage1.sh

# Build stage 1 ISO (mainly for testing, installing from this CD is
# difficult and more manually as with pacstrap)

su cross ./create_cdrom.sh

# build stage 1 hard disk image (to be installed directly as the
# stage 2 build system on the target architecture)

su cross ./create_hdd.sh

#########
# STAGE 2
#########

# Assuming stage1 of the system is installed on the target architecture
# (virtual or a real machine), we have a key-based SSH connection to
# the machine and can build packages there.

# Build stage2 in $STAGE2_BUILD with the tools on the stage1 system
# and modified PKGBUILDs and patches into the target system (replacing
# stage 1 packages). Stage 1 serves now the same function as the sysroot
# during cross-compilation of stage 1.

# The goal is to get a self-hosting system on the target architecture,
# not necesseraly containing all of base and base-devel, but enough to
# bootstrap itself on the target architecture.

su cross ./prepare_stage2_repo.sh

# Build stage 2 on the target architecture and install it onto the
# stage 1 system. Resulting artifacts get stored also back
# in $STAGE2_BUILD.

# Most things compile just fine with 64 MB RAM, binutils ld gold needs more
# (we gave it 512 MB)

./build_stage2.sh

#########
# STAGE 3
#########

# Use stage 2 to build base and base-devel. In this phase we don't care
# yet much about testing the packages (as this draws in still too many
# dependencies). But we try to be as close to base + base-devel as possible
# with minimal patching of PKGBUILDs.

# Also ommit most documentation building as it has heavy dependencies.

# We break cycles by making the simpler dependency vanish in the more
# complex packages:
# - libsasl without openldap
# - libgcrypt without libsecret
# - ldns unbound dnssec-anchors are all in a cyrcle
# - ca-certificates-mozilla needs gyp which need ninja which has
#   a download location only verifiable by the very same certificates :-)
# - dbus without systemd (because of a cyrcle), same goes for cryptsetup
# - glib2 without shared-mime-info

# some packages are simply to hard or important to build:
# - licenses: needs links, we don't care for stage3 as they will not
#   go public in the end
# - libsecret, calls gtkdocize in autogen.sh
# - systemd-sysvcompat: should be optional, I don't see why this should
#   be in base
# - nss: has big issues with hand-crafted assembler optimizations all
#   over the place drawing in SSE2 as a requirement, used for
#   building ca-certificates-mozilla only at the moment

# systemd: goal is to have a libsystemd we can link against, but stage 3
# will not have a running systemd init system!

su cross ./prepare_stage3_repo.sh

# Build stage 3 on the target architecture and install it onto the
# stage 2 system. Resulting artifacts get stored also back
# in $STAGE3_BUILD.

# Most things compile just fine with 64 MB RAM, binutils ld gold needs more
# (we gave it 512 MB). s-nail needs much more (we gave it 2 GB).

./build_stage3.sh

#########
# STAGE 4
#########

# Use stage 3 to build base and base-devel again. In this phase we try to
# use vanilla PKGBUILDs as much as possible, also building documentation
# and do all the testing.

# We don't go to another build system for now till we can build the base
# system without patched PKGBUILDs (at least not where not needed).

# We also try to get a running systemd in this stage.

# Ommitting the following in this stage:
# - bootstrapping issues:
#   - gcc-ada: needs a running ada to build
#   - java-runtime: nedds another java-runtime to build
#   - vala: needs another vala to build

su cross ./prepare_stage4_repo.sh

# Build stage 4 on the target architecture and install it onto the
# stage 3 system. Resulting artifacts get stored also back
# in $STAGE4_BUILD.

./build_stage4.sh

# Some software doesn't build
# - nss: but we need it only to produce a mozilla-certificates-ca, so
#   can also take the one from the host
#
# Some testing is impossible to do in stage4:
# - some python modules are virtually untestable due to excessive dependencies
#   e.g, python-packaging
# - glibc tests run out of memory