Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updates for latest kiwi ng (10.x) version changes #160

Merged
merged 2 commits into from Apr 19, 2024
Merged

Updates for latest kiwi ng (10.x) version changes #160

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5849bf5
Updates for latest kiwi ng (10.x) version changes
Hooverdan96 Mar 7, 2024
9fde0d3
Documentation syntax/grammar correction
Hooverdan96 Mar 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
81 changes: 54 additions & 27 deletions README.md
View file
Open in desktop
Expand Up @@ -14,7 +14,7 @@ The "target system" element is either generic, i.e. **x86_64** or **AArch64**, o
With the latter ARM64EFI spanning both generic (Arm64) and specific (64 bit EFI).

## Core Profiles
Our current pre-built installers, see: [Downloads](https://rockstor.com/dls.html), are built using the following profiles:
Our current pre-built installers are built using the following profiles (see: [Downloads](https://rockstor.com/dls.html)):

- **Leap15.5.x86_64**
- **Leap15.5.RaspberryPi4** (supports RPi400 also)
Expand Down Expand Up @@ -46,16 +46,10 @@ The resulting installer is intended to support 64-bit ARM systems that implement
Note that additional drives may be required for your specific hardware, if so consider [Installing the Stable Kernel Backport](https://rockstor.com/docs/howtos/stable_kernel_backport.html).
Also see the following subsection for enabling these same repositories/facilities within the resulting installer itself.

### Stable Kernel Backport & matching btrfs-progs
Note that we have, remarked out, the 'Stable Kernel Backport' and 'filesystems' repositories within our `rockstor.kiwi` config.
This enables building a custom installer with these back-ported modifications out-of-the-box.
See [Installing the Stable Kernel Backport](https://rockstor.com/docs/howtos/stable_kernel_backport.html) for more information.
Note that this is not a supported configuration so do indicate this modification if reporting issues on our forum or GitHub.

## HOWTO

Please see the [overview](https://osinside.github.io/kiwi/overview.html) section of the kiwi-ng docs for the canonical
[System Requirements](https://osinside.github.io/kiwi/overview.html#system-requirements) for building the installer: e.g. 15 GB free space, Python 3.5+ etc.
[System Requirements](https://osinside.github.io/kiwi/overview.html#system-requirements) for building the installer: e.g., 15 GB free space, Python version, etc.

Given our image target OS is exclusively 'Built on openSUSE', a vanilla openSUSE Leap 15.5 install is recommended if not using the kiwi-ng boxbuild method.
But if the newer kiwi-ng boxbuild method in "Building on any linux host... " is used, any relatively modern linux system can be used to build the installer.
Expand All @@ -78,28 +72,39 @@ Now you just need the kiwi-ng program this config requires to make the final ins
### Building on any linux host (KVM support required)
Kiwi now has support for using KVM virtual machines to build in an isolated environments, regardless of the linux host.

On any linux platform with KVM virtualization enabled, and python3, you can use an isolated python virtual environment to build the installer without modifying your host OS,
With the release of version 10.x of `kiwi ng` the minimum required python version is now `3.9` or higher. On any linux platform with KVM virtualization enabled,
and the corresponding python3 version, you can use an isolated python virtual environment to build the installer without modifying your host OS,
or needing to manually set up an openSUSE virtual machine (see below for this alternative).
This boxbuild approach does have some overheads compared to building on a baremetal installation, but on reasonably powerful hardware it can still take less than twenty minutes.

By default, the kiwi-boxed-plugin will reserve 8GB of memory, and 4 CPU cores.
This can be modified with `--box-memory=<vm>G --box-smp-cpus=<number>`, e.g. `--box-memory 4G --box-smp-cpus=2`.
This can be modified with `--box-memory=<vm>G --box-smp-cpus=<number>`, e.g., `--box-memory 4G --box-smp-cpus=2`.
On machines with low RAM, building with as low as 1GB has been tested successfully.
Assigning too much ram will crash your host, so be sure to set a safe amount smaller than your current available host ram.
Assigning too much RAM will crash your host, so be sure to set a safe amount smaller than your current available host ram.
Arguments before the `--` are passed to boxbuild, and after are passed to the kiwi-ng build itself.

**Note on using a host OS within a Virtual Machine:**
If using a Linux OS in a Virtual Machine, any shared folders from the host are not properly recognized by the KVM/QEMU that is generated by `kiwi-ng`'s boxbuild approach.
This can lead to `chroot` errors and terminate the installer generation.
The recommendation in this case is to clone the installer git directly into the VM directory and edit the relevant files (e.g. `rockstor.kiwi`) before executing the box build; or edit them outside of the VM and transfer them back into the VM's installer directory using a shared folder.
The recommendation in this case is to clone the installer git directly into the VM directory and edit the relevant files (e.g., `rockstor.kiwi`) before executing the box build; or edit them outside of the VM and transfer them back into the VM's installer directory using a shared folder.
At the end of installer creation the `.iso` file must then be copied from within the VM directory back onto the VM's host for further use.

**Note on `pip` usage:**
Some distros might still have a split between Python 2.x/3.x usage of `pip` (i.e. pip3 vs. pip), or an existing system that is being used had both installed over time. If that is the case, one wants to ensure that the 3.x version is used, by explicitly declaring `pip3` in the below command line. Otherwise, this can lead to execution errors down the line.
Some distros might still have a split between Python 2.x/3.x usage of `pip` (i.e. pip3 vs. pip), or an existing system that is being used had both installed over time. If that is the case, one wants to ensure that the 3.x version is used, by explicitly declaring `pip3` in the below command line.
Otherwise, this can lead to execution errors down the line.

```shell
python3 -m venv kiwi-env
```

**Note on virtual environment with a specific python3 version:**
In case multiple python3 versions are installed (e.g., 3.11 was added to enable the usage of `kiwi`), the `venv` should be created using the specific version. Otherwise, the kiwi and box-plugin versions will revert to a lower version than 10x.
Therefore, unless the higher python3 version has been set up to be the default version, the virtual environment should be created (using the example of version `3.11`):

```shell
python3.11 -m venv kiwi-env
```

If the system/distro has dropped python 2.x support, or if it is not installed on the system that is used for the build:
```shell
./kiwi-env/bin/pip install kiwi kiwi-boxed-plugin
Expand All @@ -121,16 +126,25 @@ This was the preferred method before the above kiwi-ng boxbuild capability exist
For an openSUSE Leap 15.5 OS from kiwi-ng's doc [Installation](https://osinside.github.io/kiwi/installation.html#installation) section we have:

#### x86_64 host for x86_64 profiles
Any x86_64 machine, although keep in mind that building the ISO installer is computationally expensive so Haswell or better is recommended.
Any x86_64 machine, although keep in mind that building the ISO installer is computationally expensive, so systems with a decent-sized CPU/RAM combination
released in the last 5-7 years is recommended.

##### Leap 15.5 host
The openSUSE host version should, ideally, be at least the version of the target profile.
The openSUSE host version should ideally be at least the version of the target profile. Since Leap 15.5 ships with a `python 3.6x` it is necessary
to install a higher python version (e.g., `3.11`):

```shell
sudo zypper in python311
```

then add the required repository and install kiwi and the additional requirements as shown below:

```shell
sudo zypper addrepo http://download.opensuse.org/repositories/Virtualization:/Appliances:/Builder/openSUSE_Leap_15.5/ appliance-builder
sudo zypper install python3-kiwi btrfsprogs gfxboot qemu-tools gptfdisk e2fsprogs squashfs xorriso dosfstools
```

#### AArch64 host (e.g. a Pi4) for AArch64 profiles
#### AArch64 host (e.g., a Pi4) for AArch64 profiles
See [HCL:Raspberry Pi4](https://en.opensuse.org/HCL:Raspberry_Pi4).
Install, for example, an appliance JeOS Leap 15.5 image as the host OS.
Enabling USB boot on older Pi4 systems will allow for the use of, for example, an SSD as the system drive which will massively speed up installer building.
Expand All @@ -139,7 +153,14 @@ USB booting on the Pi4 may require a bootloader update via a fully updated Raspb
Pi4 EEPROM/bootloader version "Jun 15 2020" or later will be required for USB boot regardless of any installer /EFI file changes.

##### For a JeOS Leap 15.5 host:
(Leap 15.3 and higher has moved to mixed X86_64/aarch64 repos)
(Leap 15.3 and higher has moved to mixed X86_64/aarch64 repos). Since Leap 15.5 ships with a `python 3.6x` it is necessary
to install a higher python version (e.g., `3.11`):

```shell
sudo zypper in python311
```

then add the required repository and install kiwi and the additional requirements as shown below:

```shell
sudo zypper addrepo https://download.opensuse.org/repositories/Virtualization:/Appliances:/Builder/openSUSE_Leap_15.5/ appliance-builder
Expand All @@ -151,6 +172,12 @@ No edit is required if you wish to use the generic installer filename and defaul
To change these defaults edit all lines directly preceded by **<!--Change to ...** as per the **...** details given.
Our release infrastructure performs these same edits to set official installer filenames and rockstor package versions.

#### Stable Kernel Backport & matching btrfs-progs
To enable the use of 'Stable Kernel Backport' and 'filesystems' repositories within our `rockstor.kiwi` config uncomment the
corresponding repositories already put into the file. For more information about using the most recent kernels
see [Installing the Stable Kernel Backport](https://rockstor.com/docs/howtos/stable_kernel_backport.html) for more information.

#### Root disk LUKS encryption
If you want to enable LUKS encryption of the Root disk (where Rockstor is installed), uncomment the relevant parameters
that are available in the **Leap15.5.x86_64** profile which are preceded by relevant comments. This will enable `luks2` encryption
and utilize PBKDF2, as grub does not yet support the more recent `argon2id` algorithm.
Expand All @@ -167,30 +194,30 @@ kiwi-ng --profile=Leap15.5.x86_64 --type oem system build --description ./ --tar
### Leap15.5.RaspberryPi4 profile
Executed, as the root user, in the directory containing this repository's `rockstor.kiwi` file.
N.B. RPi400 built-in keyboard is known not to work with this profile.

```shell
kiwi-ng --profile=Leap15.5.RaspberryPi4 --type oem system build --description ./ --target-dir /home/kiwi-images/
```

## Resulting Rockstor installers
With the above suggested kiwi-ng commands the resulting installers will be found in **/home/kiwi-images/** on the kiwi-ng host systems.
With the above suggested `kiwi-ng` commands the resulting installers will be found in **/home/kiwi-images/** on the kiwi-ng host systems.

- For the x86_64 profiles the resulting installer is an ISO image intended for image transfer to an installer only device.
Use the file ending in ".iso".
- For the RaspberryPi4 profile the resulting installer is an uncompressed raw disk image intended for image transfer to the target system disk directly.
Use the file ending in ".raw".

The resulting installs will grow, on first boot, to the size of their host devices.
The resulting installs will grow on first boot to the size of their host devices.
All partitioning is fully automatic.

Please see [Rockstor’s “Built on openSUSE” installer](https://rockstor.com/docs/installer-howto/installer-howto.html) HowTo for a user guide.

## Help or Assistance
Our [friendly forum](https://forum.rockstor.com/) is a good place to ask question regarding this HowTo.
Please be patient however as our support is predominantly community based and the above procedure is not that well known by our community members.
If you find an issue with this procedure and its associated kiwi-ng config, please consider submitting a pull request with any fixes or improvements you consequently discover or invent.
Our [friendly forum](https://forum.rockstor.com/) is a good place to ask question regarding this HowTo. If you enable some of the above described enhancements
(like backported kernel, LUKS, or add other options yourself) please point those out when reporting any problems.
Please be patient however, as our support is predominantly community based and the above procedure is not that well known by our community members.
If you find an issue with this procedure and its associated kiwi-ng config, please consider creating and issue and submitting a pull request with any fixes
or improvements you consequently discover or invent.

'The Rockstor Project' is a community endeavour and depends on [update channel](https://rockstor.com/docs/update-channels/update_channels.html)
subscriptions and contributions for it's continued open source development.




subscriptions and contributions for its continued open source development.
6 changes: 3 additions & 3 deletions rockstor.kiwi
View file
Open in desktop
Expand Up @@ -84,7 +84,6 @@ https://build.opensuse.org/package/show/openSUSE:Leap:15.5:Images/JeOS
initrd_system="dracut"
filesystem="btrfs"
fsmountoptions="noatime"
bootloader="grub2"
firmware="efi"
installiso="true"
kernelcmdline="nomodeset plymouth.enable=0 rd.kiwi.oem.maxdisk=5000G"
Expand All @@ -94,6 +93,7 @@ https://build.opensuse.org/package/show/openSUSE:Leap:15.5:Images/JeOS
btrfs_quota_groups="false"
efipartsize="64"
>
<bootloader name="grub2"/>
<!-- For full disk LUKS encryption uncomment below entries -->
<!-- PBKDF2 is required for grub to recognize encryption -->
<!-- <luksformat> -->
Expand Down Expand Up @@ -144,7 +144,6 @@ https://build.opensuse.org/package/show/openSUSE:Leap:15.5:Images/JeOS
initrd_system="dracut"
filesystem="btrfs"
fsmountoptions="noatime,compress=lzo"
bootloader="grub2"
firmware="efi"
kernelcmdline="plymouth.enable=0 rd.kiwi.oem.maxdisk=5000G console=ttyS0,115200 console=tty"
bootpartition="false"
Expand All @@ -154,6 +153,7 @@ https://build.opensuse.org/package/show/openSUSE:Leap:15.5:Images/JeOS
efipartsize="64"
editbootinstall="editbootinstall_rpi.sh"
>
<bootloader name="grub2"/>
<systemdisk>
<volume name="home"/>
<volume name="root"/>
Expand Down Expand Up @@ -195,7 +195,6 @@ https://build.opensuse.org/package/show/openSUSE:Leap:15.5:Images/JeOS
initrd_system="dracut"
filesystem="btrfs"
fsmountoptions="noatime,compress=lzo"
bootloader="grub2"
firmware="efi"
kernelcmdline="plymouth.enable=0 rd.kiwi.oem.maxdisk=5000G earlycon"
bootpartition="false"
Expand All @@ -205,6 +204,7 @@ https://build.opensuse.org/package/show/openSUSE:Leap:15.5:Images/JeOS
efipartsize="64"
format="qcow2"
>
<bootloader name="grub2"/>
<systemdisk>
<volume name="home"/>
<volume name="root"/>
Expand Down