Skip to content

Commit

Permalink
Moved build to a script
Browse files Browse the repository at this point in the history
  • Loading branch information
kiler129 committed Feb 1, 2021
1 parent 84b52ce commit 4f151ac
Show file tree
Hide file tree
Showing 3 changed files with 158 additions and 36 deletions.
40 changes: 4 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ highly recommended.

### Installation

#### Proxmox - premade packages
#### Proxmox - premade packages (easy)
As I believe in *[eating your own dog food](https://en.wikipedia.org/wiki/Eating_your_own_dog_food)* I run the kernel
described here. Thus, I publish precompiled packages.

Expand All @@ -60,41 +60,9 @@ described here. Thus, I publish precompiled packages.

---

#### Proxmox - building from sources
If you're running a version of Proxmox with [no packages available](README.md#proxmox---premade-packages) you can
compile the kernel yourself using patches provided.

1. Prepare [at least 60GB of free disk space](https://forum.level1techs.com/t/linux-debian-proxmox-recompile-needing-over-60gb-and-counting-to-compile/160009)
2. Install required packages:
```shell script
apt update
apt install git nano screen patch fakeroot build-essential devscripts libncurses5 libncurses5-dev libssl-dev bc flex bison libelf-dev libaudit-dev libgtk2.0-dev libperl-dev asciidoc xmlto gnupg gnupg2 rsync lintian debhelper libdw-dev libnuma-dev libslang2-dev sphinx-common asciidoc-base automake cpio dh-python file gcc kmod libiberty-dev libpve-common-perl libtool perl-modules python-minimal sed tar zlib1g-dev lz4
```
3. Download everything:
```shell script
mkdir proxmox-kernel ; cd proxmox-kernel
git clone --depth=1 -b pve-kernel-5.4 git://git.proxmox.com/git/pve-kernel.git
git clone --depth=1 https://github.com/kiler129/relax-intel-rmrr.git
```
4. Add kernel patch & patch the toolchain
```shell script
cd pve-kernel
cp ../relax-intel-rmrr/patches/add-relaxable-rmrr-below-5_8.patch ./patches/kernel/CUSTOM-add-relaxable-rmrr.patch
patch -p1 < ../relax-intel-rmrr/patches/proxmox.patch
```
5. Compile the kernel
```shell script
make
```
This step will take a lot of time (30m-3h depending on your machine).

6. Install new kernel:
```shell script
dpkg -i *.deb
```
7. *(OPTIONAL)* Verify the kernel works with the patch disabled by rebooting and checking if `uname -r` shows a version
ending with `-pve-relaxablermrr`
8. [Configure the kernel](README.md#configuration)
#### Proxmox - building from sources (advanced)
If you're running a version of Proxmox with [no packages available](README.md#proxmox---premade-packages-easy) you can
[compile the kernel yourself using patches provided](build/proxmox/README.md).

---

Expand Down
35 changes: 35 additions & 0 deletions build/proxmox/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
## Proxmox - building from sources

If you're running a version of Proxmox with [no packages available](../../README.md#proxmox---premade-packages-easy), or
for some reason you don't/can't trust precompiled packages you can compile the kernel yourself using patches provided.

The easiest way to do it is to use a script provided in this repository, alongside this `README.md` file
([`build/proxmox/build.sh`](build.sh))

### Prerequisites
1. Proxmox 6 install (recommended) or Debian Buster <small>*(it WILL fail on Ubuntu!)*</small>
2. Root access
3. ~30GB of free space


### How to do it WITHOUT Docker?
This is mostly intended if you want to build & run on your Proxmox host. Jump to [Docker-ized](README.md#how-to-do-it-with-docker)
guide if you want to build packages in an isolated environment.

1. Download the [build script](build.sh) (e.g. use `wget https://raw.githubusercontent.com/kiler129/relax-intel-rmrr/build/proxmox/build.sh`)
2. Run the [`build.sh`](build.sh) script from terminal:
`RMRR_AUTOINSTALL=1 bash ./build.sh`
<small>*You can also manually execute commands in the script step-by-step. To facilitate that the script contains
extensive comments for every step.*</small>

4. *(OPTIONAL)* Verify the kernel works with the patch disabled by rebooting and checking if `uname -r` shows a version
ending with `-pve-relaxablermrr`
5. [Configure the kernel](../../README.md#configuration)

This process will leave precompiled `*.deb` packages, in case you want to copy them to other Proxmox hosts you have.


### How to do it WITH Docker?
This is mostly intended for building packages for later use (and/or when you don't want to mess with your OS).

//TODO
119 changes: 119 additions & 0 deletions build/proxmox/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
#!/usr/bin/env bash
set -e

#################################################################################
# This script is a part of https://github.com/kiler129/relax-intel-rmrr project #
#################################################################################


echo '###########################################################'
echo '############# STEP 0 - PERFORM SANITY CHECKS ##############'
echo '###########################################################'
# Make sure script is working in the directory it is located in
cd "$(dirname "$(readlink -f "$0")")"

# Build process will fail if you're not a root (+ apt actions itself need it)
if [[ "$EUID" -ne 0 ]]
then echo "This script should be run bash root"
exit
fi

# Sanity check: make sure no two builds are started nor we have something leftover from previous attempts
if [[ -d "proxmox-kernel" ]]; then
echo 'Directory "proxmox-kernel" already exists - if your previous build failed DELETE it first'
exit 1
fi


echo '###########################################################'
echo '############ STEP 1 - INSTALL ALL DEPENDENCIES ############'
echo '###########################################################'
# Check if Proxmox-specific package exists in apt cache. If it does it means apt already knows Proxmox repository, if
# not we need to add it to properly build the kernel
if apt show libpve-common-perl &>/dev/null; then
echo "Step 1.0: Proxmox repository already present - not adding"
else
# Add Proxmox repo & their signing key
echo "Step 1.0: Adding Proxmox apt repository..."
apt -y update
apt -y install gnupg
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7BF2812E8A6E88E0
echo 'deb http://download.proxmox.com/debian/pve buster pve-no-subscription' > /etc/apt/sources.list.d/pve.list
fi

# Install all packages required to build the kernel & create *.deb packages for installation
echo "Step 1.1: Installing build dependencies..."
apt -y update
apt -y install git nano screen patch fakeroot build-essential devscripts libncurses5 libncurses5-dev libssl-dev bc \
flex bison libelf-dev libaudit-dev libgtk2.0-dev libperl-dev asciidoc xmlto gnupg gnupg2 rsync lintian debhelper \
libdw-dev libnuma-dev libslang2-dev sphinx-common asciidoc-base automake cpio dh-python file gcc kmod libiberty-dev \
libpve-common-perl libtool perl-modules python-minimal sed tar zlib1g-dev lz4 curl



echo '###########################################################'
echo '############ STEP 2 - DOWNLOAD CODE TO COMPILE ############'
echo '###########################################################'
# Create working directory
echo "Step 2.0: Creating working directory"
mkdir proxmox-kernel
cd proxmox-kernel

# Clone official Proxmox kernel repo & Relaxed RMRR Mapping patch
echo "Step 2.1: Downloading Proxmox kernel toolchain & patches"
git clone --depth=1 -b pve-kernel-5.4 git://git.proxmox.com/git/pve-kernel.git
git clone --depth=1 https://github.com/kiler129/relax-intel-rmrr.git

# Go to the actual Proxmox toolchain
cd pve-kernel

# (OPTIONAL) Download flat copy of Ubuntu Focal kernel submodule
# If you skip this the "make" of Proxmox kernel toolchain will download a copy (a Proxmox kernel is based on Ubuntu
# Focal kernel). However, it will download it with the whole history etc which takes A LOT of space (and time). This
# bypasses the process safely.
# This curl skips certificate validation because Proxmox GIT WebUI doesn't send Let's Encrypt intermediate cert
echo "Step 2.2: Downloading base kernel"
curl -k "https://git.proxmox.com/?p=mirror_ubuntu-focal-kernel.git;a=snapshot;h=$(git submodule status submodules/ubuntu-focal | cut -c 2-41);sf=tgz" --output kernel.tgz
tar -xf kernel.tgz -C submodules/ubuntu-focal/ --strip 1
rm kernel.tgz



echo '###########################################################'
echo '################# STEP 3 - CREATE KERNEL ##################'
echo '###########################################################'
echo "Step 3.0: Applying patches"
cp ../relax-intel-rmrr/patches/add-relaxable-rmrr-below-5_8.patch ./patches/kernel/CUSTOM-add-relaxable-rmrr.patch
patch -p1 < ../relax-intel-rmrr/patches/proxmox.patch


echo "Step 3.1: Compiling kernel... (it will take 30m-3h)"
# Note: DO NOT add -j to this make, see https://github.com/kiler129/relax-intel-rmrr/issues/1
# This step will compile kernel & build all *.deb packages as Proxmox builds internally
make


echo '###########################################################'
echo '################ STEP 4 - INSTALL KERNEL ##################'
echo '###########################################################'
echo "Step 4: Installing packages"

if [[ -v RMRR_AUTOINSTALL ]]; then
apt install ./*.deb
else
echo '=====>>>> SKIPPED - to enable autoinstallation set "RMRR_AUTOINSTALL" environment variable.'
echo '=====>>>> To install execute "dpkg -i *.deb" after this script finishes'
fi

echo '###########################################################'
echo '################## STEP 5 - CLEANUP #######################'
echo '###########################################################'
# Remove all (~30GB) of stuff leftover after compilation
echo "Step 5: Cleaning up..."
cd ..
mkdir debs
mv pve-kernel/*.deb ./debs
rm -rf pve-kernel
rm -rf relax-intel-rmrr

exit 0

0 comments on commit 4f151ac

Please sign in to comment.