More documentation work (#1819)

* More documentation work Signed-off-by: Fredrik Lönnegren <fredrik.lonnegren@suse.com> * Improve multi-arch handling GRUB_ARCH can be calculated in runtime, this makes toolkit Dockerfile and green-example Dockerfile able to be built with buildx --platform=linux/amd64,linux/arm64 flag at the same time. Signed-off-by: Fredrik Lönnegren <fredrik.lonnegren@suse.com> --------- Signed-off-by: Fredrik Lönnegren <fredrik.lonnegren@suse.com>
rancher · Aug 31, 2023 · 2360815 · 2360815
1 parent 6c3b71b
commit 2360815
Show file tree

Hide file tree

Showing 22 changed files with 84 additions and 260 deletions.
diff --git a/.github/workflows/build_and_test.yaml b/.github/workflows/build_and_test.yaml
@@ -168,6 +168,7 @@ jobs:
       - if: ${{ env.ARCH == 'x86_64' }} 
         name: Run VM script dependencies
         run: |
+          brew update; brew upgrade qemu
           brew install bash coreutils
       - if: ${{ env.ARCH == 'x86_64' }} 
         name: Prepare test (x86_64)

diff --git a/Dockerfile b/Dockerfile
@@ -16,9 +16,6 @@ ADD internal internal
 ADD pkg pkg
 ADD main.go .
 
-# Install cosign
-RUN go install github.com/sigstore/cosign/v2/cmd/cosign@latest && cp $(go env GOPATH)/bin/cosign /usr/bin/cosign
-
 # Set arg/env after go mod download, otherwise we invalidate the cached layers due to the commit changing easily
 ARG ELEMENTAL_VERSION=0.0.1
 ARG ELEMENTAL_COMMIT=""
@@ -36,10 +33,9 @@ FROM opensuse/leap:$LEAP_VERSION AS elemental
 ARG ELEMENTAL_COMMIT=""
 ENV ELEMENTAL_COMMIT=${ELEMENTAL_COMMIT}
 
-ARG GRUB_ARCH
-ENV GRUB_ARCH=${GRUB_ARCH}
-
-RUN zypper install -y --no-recommends xfsprogs \
+RUN ARCH=$(uname -m); \
+    if [[ $ARCH == "aarch64" ]]; then ARCH="arm64"; fi; \
+    zypper install -y --no-recommends xfsprogs \
         parted \
         util-linux-systemd \
         e2fsprogs \
@@ -48,14 +44,14 @@ RUN zypper install -y --no-recommends xfsprogs \
         rsync \
         grub2 \
         dosfstools \
-        grub2-${GRUB_ARCH}-efi \
+        grub2-${ARCH}-efi \
         squashfs \
         mtools \
         xorriso \
+	cosign \
         lvm2
 
 COPY --from=elemental-bin /usr/bin/elemental /usr/bin/elemental
-COPY --from=elemental-bin /usr/bin/cosign /usr/bin/cosign
 
 # Fix for blkid only using udev on opensuse
 RUN echo "EVALUATE=scan" >> /etc/blkid.conf

diff --git a/Makefile b/Makefile
@@ -5,7 +5,6 @@ QCOW2?=$(shell ls $(ROOT_DIR)/build/*.qcow2 2> /dev/null)
 ISO?=$(shell ls $(ROOT_DIR)/build/*.iso 2> /dev/null)
 FLAVOR?=green
 ARCH?=$(shell uname -m)
-GRUB_ARCH?=$(ARCH)
 PLATFORM?=linux/$(ARCH)
 IMAGE_SIZE?=20G
 PACKER_TARGET?=qemu.elemental-${ARCH}
@@ -23,10 +22,6 @@ LDFLAGS    := -w -s
 LDFLAGS += -X "github.com/rancher/elemental-toolkit/internal/version.version=${GIT_TAG}"
 LDFLAGS += -X "github.com/rancher/elemental-toolkit/internal/version.gitCommit=${GIT_COMMIT}"
 
-ifeq ("$(GRUB_ARCH)", "aarch64")
-GRUB_ARCH=arm64
-endif
-
 # default target
 .PHONY: all
 all: build build-cli
@@ -39,7 +34,7 @@ include make/Makefile.test
 
 .PHONY: build
 build:
-	$(DOCKER) build --platform ${PLATFORM} --build-arg GRUB_ARCH=$(GRUB_ARCH) --build-arg ELEMENTAL_VERSION=${GIT_TAG} --build-arg ELEMENTAL_COMMIT=${GIT_COMMIT} --target elemental -t ${TOOLKIT_REPO}:${VERSION} .
+	$(DOCKER) build --platform ${PLATFORM} ${DOCKER_ARGS} --build-arg ELEMENTAL_VERSION=${GIT_TAG} --build-arg ELEMENTAL_COMMIT=${GIT_COMMIT} --target elemental -t ${TOOLKIT_REPO}:${VERSION} .
 
 .PHONY: push-toolkit
 push-toolkit:
@@ -51,7 +46,7 @@ build-cli:
 
 .PHONY: build-os
 build-os: build
-	$(DOCKER) build examples/$(FLAVOR) --platform ${PLATFORM} --build-arg GRUB_ARCH=$(GRUB_ARCH) --build-arg TOOLKIT_REPO=$(TOOLKIT_REPO) --build-arg VERSION=$(VERSION) --build-arg REPO=$(REPO) -t $(REPO):$(VERSION)
+	$(DOCKER) build examples/$(FLAVOR) --platform ${PLATFORM} --build-arg TOOLKIT_REPO=$(TOOLKIT_REPO) --build-arg VERSION=$(VERSION) --build-arg REPO=$(REPO) -t $(REPO):$(VERSION)
 
 .PHONY: push-os
 push-os:

diff --git a/docs/content/en/docs/Creating derivatives/build_disk.md b/docs/content/en/docs/Creating derivatives/build_disk.md
@@ -2,7 +2,7 @@
 title: "Build disk images with Elemental"
 linkTitle: "Build disk images with Elemental"
 weight: 4
-date: 2022-03-03
+date: 2023-08-31
 description: >
   This section documents the procedure to build disk images using elemental
 ---
@@ -17,13 +17,6 @@ The suggested approach is based on using the Elemental installer (`elemental ins
 from a Linux to a loop device. The loop device can be a raw image created with `qemu-img create` that can easily be
 converted to other formats after the installation by using `qemu-img convert`.
 
-## Get Elemental
-
-Elemental binary can be downloaded from the [github releases](https://github.com/rancher/elemental-cli/releases/latest) page.
-
-The golang binary can be used as is, however take into account that some linux utilities are expected to be present in the host. More
-specific elemental expects to find common linux utilities to operate over block devices: rsync, parted, blkid, lsblk, udevadm, resize2fs, tune2fs, mkfs.ext2, etc.
-
 ## Prepare the loop device
 
 Preparing the a loop device for the installation is simple and straight forward.

diff --git a/docs/content/en/docs/Creating derivatives/build_iso.md b/docs/content/en/docs/Creating derivatives/build_iso.md
@@ -2,35 +2,31 @@
 title: "Build ISOs"
 linkTitle: "Build ISOs"
 weight: 4
-date: 2017-01-05
+date: 2023-08-31
 description: >
   Build ISOs from bootable images
 ---
 
-![](https://docs.google.com/drawings/d/e/2PACX-1vReZtyNs0imrji-AwnqK0-4ekCKLcKzfnQ_CwiMj93Q7IsycAJHwlNohwCv_hyHnaify7qO-v2Cecg5/pub?w=1223&h=691)
+In order to build an ISO we rely on `elemental build-iso` command. It accepts a YAML file denoting the sources to bundle in an ISO. In addition it can also overlay custom files or use container images from a registry as packages.
 
-In order to build an ISO we rely on [elemental build-iso](https://github.com/rancher/elemental-cli) command. It accepts a YAML file denoting the sources to bundle in an ISO. In addition it can also overlay custom files or use container images from a registry as packages.
-
-To build an iso, just run:
+To build an ISO, just run:
 
 ```bash
-docker run --rm -ti -v $(pwd):/build quay.io/costoolkit/elemental-cli:v0.0.14-e4e39d4 --debug build-iso -o /build $SOURCE
+docker run --rm -ti -v $(pwd):/build ghcr.io/rancher/elemental-toolkit/elemental-cli:latest --debug build-iso -o /build $SOURCE
 ```
 
 Where `$SOURCE` might be the container image you want to build the ISO for, you might want to check on [how to build bootable images](../creating_bootable_images). Argument `$SOURCE` might be the reference to the directory, file, container image or channel we are building the ISO for, it should be provided as uri in following format <sourceType>:<sourceName>, where:
     * <sourceType> - might be ["dir", "file", "oci", "docker"], as default is taken "docker"
     * <sourceName> - is path to file or directory, channel or image name with tag version (if tag was not provided then "latest" is used)
 
-Some examples for $SOURCE argument "dir:/cOS/system", "oci:quay.io/repository/costoolkit/releases-green:cos-system-0.8.14-10", "channel:system/cos"
-
 `elemental build-iso` command also supports reading a configuration `manifest.yaml` file. It is loaded form the directory specified by `--config-dir` elemental's flag.
 
 An example of a yaml file using the bootloader from the contained image:
 
 ```yaml
 iso:
   bootloader-in-rootfs: true
-  label: "COS_LIVE"
+  grub-entry-name: "Installer"
 
 name: "Elemental-0"
 date: true

diff --git a/docs/content/en/docs/Creating derivatives/cosign.md b/docs/content/en/docs/Creating derivatives/cosign.md
diff --git a/docs/content/en/docs/Customizing/embedded_features.md b/docs/content/en/docs/Customizing/embedded_features.md
@@ -0,0 +1,32 @@
+---
+title: "Embedded configuration"
+linkTitle: "Embedded configuration"
+weight: 3
+date: 2023-08-31
+description: >
+  Extracting default system configuration
+---
+
+Elemental-toolkit provides some default configuration files for the following components:
+
+- GRUB2
+- Dracut
+- Cloud init files
+- Boot assessment
+
+These configuration files can be installed into a Derivative using the `elemental init`-command
+
+The `init`-command should be used inside the Dockerfile as in the following example:
+
+{{<githubembed repo="rancher/elemental-toolkit" file="examples/green/Dockerfile" lang="Dockerfile">}}
+
+The current features available for the `init`-command is:
+
+- immutable-rootfs: dracut configuration for mounting the immutable root filesystem.
+- grub-config: grub configuration for booting the derivative.
+- elemental-setup: services used for booting the system and running cloud-init files at boot/install/upgrade.
+- dracut-config: default dracut configuration for generating an initrd.
+- cloud-config-defaults: optional default settings for a derivative.
+- cloud-config-essentials: essential cloud-init files.
+
+
diff --git a/docs/content/en/docs/Customizing/general_configuration.md b/docs/content/en/docs/Customizing/general_configuration.md
@@ -18,7 +18,7 @@ By default `<config-dir>` is set to `/etc/elemental` however this can be changed
 
 Below you can find an example of the config file including most of the available options:
 
-{{<githubembed repo="rancher/elemental-cli" file="config.yaml.example" lang="yaml">}}
+{{<githubembed repo="rancher/elemental-toolkit" file="config.yaml.example" lang="yaml">}}
 
 
 The `system` and `recovery-system` objects are an image specification. An image specification is defined by:

diff --git a/docs/content/en/docs/Customizing/login.md b/docs/content/en/docs/Customizing/login.md
@@ -3,12 +3,12 @@
 title: "Login"
 linkTitle: "Login"
 weight: 3
-date: 2017-01-05
+date: 2023-08-31
 description: >
   Default login, and how to override it
 ---
 
-By default you can login with the user `root` and `cos` in a vanilla Elemental image, this is also set automatically by the {{<package package="system/cloud-config" >}} package if used by a derivative.
+By default you can login with the user `root` and `cos` in a vanilla Elemental image, this is also set automatically by the `cloud-config-defaults` feature if used by a derivative.
 
 You can change this by overriding `/system/oem/04_accounting.yaml` in the container image if present, or via [cloud-init](../../reference/cloud_init/#stagesstage_idstep_nameusers).
 

diff --git a/docs/content/en/docs/Customizing/oem_configuration.md b/docs/content/en/docs/Customizing/oem_configuration.md
@@ -2,7 +2,7 @@
 title: "OEM configuration"
 linkTitle: "OEM configuration"
 weight: 3
-date: 2017-01-05
+date: 2023-08-31
 description: >
   OEM configuration reserved to Elemental and derivatives
 ---
@@ -16,15 +16,11 @@ For runtime persistence configuration, the only supported way is with cloud-conf
 
 A derivative automatically loads and executes cloud-config files during the various system [stages](../stages) also inside `/system/oem` which is read-only and reserved to the system.
 
-{{% alert title="Note" %}}
-The cloud-config mechanism works also as an emitter event pattern - running services or programs can emit new custom `stages` in runtime by running `cos-setup stage_name`.
-{{% /alert %}}
-
 Derivatives that wish to override default configurations can do that by placing extra cloud-init file, or overriding completely `/system/oem` in the target image.
 
-This is to setup for example, the default root password or the prefered upgrade channel. 
+This is to setup for example, the default root password or the preferred upgrade channel. 
 
-The following are the `Elemental` default oem files, which are shipped within the {{<package package="system/cloud-config" >}}, which is an optional package:
+The following are the `Elemental` default oem files, which are shipped within the `cloud-config-defaults` and `cloud-config-essentials` features:
 
 ```
 /system/oem/00_rootfs.yaml - defines the rootfs mountpoint layout setting
@@ -36,4 +32,4 @@ The following are the `Elemental` default oem files, which are shipped within th
 /system/oem/06_recovery.yaml - Executes additional commands when booting in recovery mode
 ```
 
-You can either override the above files, or alternatively not consume the {{<package package="system/cloud-config" >}} package while building a derivative.
+You can either override the above files, or alternatively not consume the above features while building a derivative.
diff --git a/docs/content/en/docs/Customizing/runtime_persistent_changes.md b/docs/content/en/docs/Customizing/runtime_persistent_changes.md
@@ -2,7 +2,7 @@
 title: "Runtime persistent changes"
 linkTitle: "Runtime persistent changes"
 weight: 3
-date: 2021-09-24
+date: 2023-08-31
 description: >
   Applying changes to Elemental images in runtime or “how to install a package in an immutable OS at runtime?”
 ---
@@ -19,8 +19,6 @@ To apply transient changes, it's possible to boot a Elemental derivative in read
 
 ## Persist changes with Cloud init files
 
-Note: The following applies only to derivatives with {{<package package="utils/installer" >}} at version `0.17` or newer
-
 Elemental allows to apply a set of commands, or cloud-init steps, during upgrade, deploy, install and reset in the context of the target image, in RW capabilities. This allows to carry on changes during upgrades on the target image without the need to re-build or have a custom derivative image.
 
 All the configuration that we want to apply to the system will run each time we do an upgrade, a reset or an installation on top of the new downloaded image (in case of upgrade) or the image which is the target system. 

diff --git a/docs/content/en/docs/Customizing/selinux_support.md b/docs/content/en/docs/Customizing/selinux_support.md
@@ -23,7 +23,7 @@ Note it is required to load the policy in advance to be capable to apply the `re
 
 ## Using custom SELinux modules
 
-Making use of `selinux` and including SELinux utilities and targeted policy within the base OS it is enough to get started with SELinux, however there is a great chance that this is too generic and requires some additional policy modules to be fully functional according to each specific use case. In that regard the  `system/elemental-selinux` luet package provides a basic custom module for illustration purposes, **it should NOT be used as is** in production. `system/elemental-selinux` provides the sources of a very simple Type Enforcement (TE) file (`/usr/share/elemental/selinux/elemental.te`) and the compiled Policy Package (`/usr/share/elemental/selinux/elemental.pp`) out of the TE file, ready to be installed and loaded by SELinux.
+Making use of `selinux` and including SELinux utilities and targeted policy within the base OS it is enough to get started with SELinux, however there is a great chance that this is too generic and requires some additional policy modules to be fully functional according to each specific use case.
 
 The Type Enforcement file was created by booting an Elemental OS on permissive mode using `audit2allow` and other SELinux related utilities to generate the custom module out of the reported denials. Something like:
 

diff --git a/docs/content/en/docs/Customizing/stages.md b/docs/content/en/docs/Customizing/stages.md
@@ -14,8 +14,6 @@ We have a custom augmented cloud-init syntax that allows to hook into various st
 - Network availability
 - During upgrades, installation, deployments  , and resets
 
-![Stages](https://docs.google.com/drawings/d/e/2PACX-1vRuITNgkCeDfS4LqxDvz2j4WxuRDzU8dJOTa5CY88ya8_hJ1QeaGKipTanggtiXiCxhwkUpYld-Cbxa/pub?w=885&h=761)
-
 Cloud-init files in `/system/oem`, `/oem` and `/usr/local/oem` are applied in 5 different phases: `boot`, `network`, `fs`, `initramfs` and `reconcile`. All the available cloud-init keywords can be used in each stage. Additionally, it's possible also to hook before or after a stage has run, each one has a specific stage which is possible to run steps: `boot.after`, `network.before`, `fs.after` etc.
 
 Multiple stages can be specified in a single cloud-init file.

diff --git a/docs/content/en/docs/Examples/creating_bootable_images.md b/docs/content/en/docs/Examples/creating_bootable_images.md
@@ -7,7 +7,6 @@ description: >
   This document describes the requirements to create standard container images that can be used for `Elemental` deployments
 ---
 
-
 You can find the examples below in the [examples](https://github.com/rancher/elemental-toolkit/tree/main/examples) folder.
 
 ## From standard images
@@ -16,7 +15,6 @@ Besides using the `elemental-toolkit` toolchain, it's possible to create standar
 
 An example of a Dockerfile image can be:
 
-
 {{<githubembed repo="rancher/elemental-toolkit" file="examples/green/Dockerfile" lang="Dockerfile">}}
 
 We can just run docker to build the image with 
@@ -34,6 +32,8 @@ kernel
 dracut
 ```
 
+And then extract the configuration for the system using the `elemental init`-command.
+
 ## Customizations
 
 All the method above imply that the image generated will be the booting one, there are however several configuration entrypoint that you should keep in mind while building the image:

diff --git a/docs/content/en/docs/Examples/embedded_images.md b/docs/content/en/docs/Examples/embedded_images.md
@@ -27,7 +27,7 @@ $> cd derivative
 Let's create now a `Dockerfile` for our image inside that directory, which will be represent running system:
 
 ```Dockerfile
-FROM ghcr.io/rancher/elemental-toolkit/elemental-cli:v0.10.7 AS elemental
+FROM ghcr.io/rancher/elemental-toolkit/elemental-cli:v0.11.0 AS elemental
 
 FROM registry.suse.com/suse/sle-micro-rancher/5.2
 ARG K3S_VERSION=v1.20.4+k3s1

diff --git a/docs/content/en/docs/Getting started/download.md b/docs/content/en/docs/Getting started/download.md
@@ -3,42 +3,32 @@
 title: "Download"
 linkTitle: "Download"
 weight: 1
-date: 2017-01-05
+date: 2023-08-31
 description: >
-  How to get Elemental assets: ISOs, Cloud Images ...
+  How to get Elemental Toolkit
 ---
 
-Elemental-toolkit releases consist on container images that can be used to build derived against and the cos source tree itself.
-
-Elemental supports different release channels, all the final and cache images used are tagged and pushed regularly [to Quay Container Registry](https://quay.io/repository/costoolkit/releases-teal) and can be pulled for inspection from the registry as well.
-
-Those are exactly the same images used during upgrades, and can also be used to build Linux derivatives from Elemental.
+Elemental-toolkit consists of a CLI program that is used to install a system and build bootable sources. The CLI also embeds configuration needed for a bootable derivative.
 
 ## Download Elemental
 
-You can also try out Elemental from the vanilla images and use it to experiment locally or either bootstrap a derivative: those are minimal system with a small package set in order to boot and deploy a container. 
-
-Latest Elemental-toolkit releases assets (ISOs, Raw disks, Cloud images) can be found on [Github](https://github.com/rancher/elemental-toolkit/releases/), check [Booting](../booting) for an explanation of each asset type and how to use it.
-
-Elemental can run in: VMs, baremetals and Cloud - the default login username/password is `root/cos`.
-
-### Install
-
-To install run `elemental install <device>` to start the installation process. Remove the ISO/medium and reboot.
-
-_Note_: `elemental install` supports other options as well. Run `elemental install --help` to see a complete help.
+Elemental toolkit can be run directly using a container runtime such as docker: 
 
-## Releases
+```bash
+docker run -it --rm ghcr.io/rancher/elemental-toolkit/elemental-cli:latest version
+```
 
-Elemental has 4 variants:
+## Building from source
 
-- [teal](https://quay.io/repository/costoolkit/releases-teal): SLE Micro for Rancher based one, shipping packages from Sle Micro 5.3.
-- [green](https://quay.io/repository/costoolkit/releases-green): openSUSE based one, shipping packages from OpenSUSE Leap 15.4 repositories.
-- [blue](https://quay.io/repository/costoolkit/releases-blue): Fedora based one, shipping packages from Fedora 33 repositories
-- [orange](https://quay.io/repository/costoolkit/releases-orange): Ubuntu based one, shipping packages form Ubuntu 20.10 repositories
+The CLI can also be built from source by checking out the repo and running make:
 
-We currently support and test only the **teal** variant.
+```bash
+git clone https://github.com/rancher/elemental-toolkit
+cd elemental-toolkit
+make build-cli
+./build/elemental version
+```
 
 ## What to do next?
 
-Check out [the customization section](../../customizing) to customize `Elemental` or [the example section](../../examples) for some already prepared recipe examples.
+Check out [the customization section](../../customizing) to build a custom `Elemental` derivative or [the example section](../../examples) for some already prepared recipe examples.