Improve uki iso stuff #1854

Itxaka · 2023-09-25T14:51:03Z

What this PR does / why we need it:

Splits UKI tests into its own file as we may need to do heavy changes to it
Create /oem and /efi in the rootfs whehn building uki artifacts
Add missing oem labels in the cmdline
Comment out the generation of signing keys for uki artifacts
Bundle some pregenerated keys to sign efi artifacts for testing
ALWAYS sign the efi artifacts, no matter if we are using secure boot or not, it doesnt hurt (NOTE: This needs to be dropped on release, follow up needed)
Sign systemd-boot with our test keys for testing
Change boot method for uki. Now the fallback for uefi is set to systemd-boot which loads a config file for kairos. That way several efi variables get set properly. Also we gain having several versions in there and fallback.
Bump peg to support detaching CD on qemu
Bundle a custom efivars file for qemu that has secureboot enabled and our keys injected so we can test secureboot directly.
Add efivars.json with the stored efivars variables so we can regenerate the efivars.fd file
Add efivars.md to explain the efivars use and regeneration
Adds tests for uki:
- boot mode (install or boot)
- secureboot enabled
- mounts missing on install mode
- mounts exist on boot mode

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Itxaka · 2023-09-25T15:47:55Z

Earthfile

@@ -495,8 +495,10 @@ uki-artifacts:
    FROM +base-image --BUILD_INITRD=false
    RUN /usr/bin/immucore version
    RUN ln -s /usr/bin/immucore /init
+    RUN mkdir -p /oem # be able to mount oem under here if found


this makes it easier to deal with mounts later

why does this only apply for UKI artifacts?

there doesnt seem to be an oem dir on the rootfs when generating the uki artifact so if we want to mount something in there on boot, the path needs to exist before. No ieda why the rootfs doesnt have oem....

but should we create it at the Dockerfile then?

No idea, I need to investigate because the isos and images DO have it...so it has to be created somewhere...

Itxaka · 2023-09-25T15:48:03Z

Earthfile

    RUN find . \( -path ./sys -prune -o -path ./run -prune -o -path ./dev -prune -o -path ./tmp -prune -o -path ./proc -prune \) -o -print | cpio -R root:root -H newc -o | gzip -2 > /tmp/initramfs.cpio.gz
-    RUN echo "console=tty1 console=ttyS0 net.ifnames=1 rd.immucore.debug rd.immucore.uki selinux=0" > /tmp/Cmdline
+    RUN echo "console=tty1 console=ttyS0 net.ifnames=1 rd.immucore.oemlabel=COS_OEM rd.immucore.oemtimeout=2 rd.immucore.debug rd.immucore.uki selinux=0" > /tmp/Cmdline


was missing the oem config

when releasing, I guess we don't want to have that immucore debug there ... I wonder if these could be passed in some other way to not have different targets depending on whether it's a release or a normal build

yes, there is a nice feature of systemd-boot in which you can have a separated efi artifact that contains the comand line only! And that one is also signed so supposedly it works with secureboot and such.

That would allow us to also have an install entry cmdline, a boot entry cmdline and a debug entry cmdline with minimal work.

Unfortunately I had no time to investigate this further but there is some work ongoing and it should be possible (but experimental) to do so: systemd/systemd#27358

Earthfile

Itxaka · 2023-09-25T15:48:44Z

Earthfile

+    COPY +uki-signed/DB.auth .
+    COPY +uki-signed/MokManager.efi .
+    # Set the name for kairos manually as otherwise it picks it from the os-release automatically
+    RUN printf "title Kairos ${FLAVOR} ${VERSION}\nefi /EFI/kairos/kairos.efi" > kairos.conf


cretes the config for systemd-boot to show in the entries

Earthfile

Itxaka · 2023-09-25T15:50:24Z

Earthfile

-    RUN mcopy -i /build/efi/efiboot.img /build/uki.efi ::EFI/BOOT/BOOTX64.EFI
-    RUN xorriso -as mkisofs -V 'EFI_ISO_BOOT' -e efiboot.img -no-emul-boot -o /build/$ISO_NAME.iso /build/efi/
+    RUN mcopy -i /build/efi/efiboot.img /usr/lib/systemd/boot/efi/systemd-bootx64.efi ::EFI/BOOT/BOOTX64.EFI
+    RUN xorriso -as mkisofs -V 'UKI_ISO_INSTALL' -e efiboot.img -no-emul-boot -o /build/$ISO_NAME.iso /build/efi/


iso name is currently used as to find the source of the install due to how uki works. Basically there is no iso mounted anywhere that we can use as source like with normal isos, and we need to access the efi file directly to copy it over and respect signatures and such so this is a needed hack for now to dev this

framework-profile.yaml

Itxaka · 2023-09-29T18:45:13Z

Earthfile

+    #    --pcr-public-key public.pem \
+    #    --measure \
+    #    --output $ISO_NAME.signed.efi
+    RUN sbsign --key DB.key --cert DB.crt --output systemd-bootx64.signed.efi /usr/lib/systemd/boot/efi/systemd-bootx64.efi


alwasy sign our efi artifacts, makes no sense to test them without signature

Signed-off-by: Itxaka <itxaka@kairos.io>

- Use test keys for development, easier to test if we always sign with the same key as we only need to insert into the EFI once - Sign systemd-boot - Also copy and create an entry for mokmanager so we can enroll keys using it if needed - Bump packages with uki fixes for layout Signed-off-by: Itxaka <itxaka@kairos.io>

Signed-off-by: Itxaka <itxaka@kairos.io>

Signed-off-by: Itxaka <itxaka.garcia@spectrocloud.com>

Signed-off-by: Itxaka <itxaka@kairos.io>

mauromorales · 2023-10-03T13:43:10Z

.github/workflows/uki.yaml

+          # Use immmucore master as it has patches not released for uki
+          # Use kairos-agent main branch as it has patches not released for uki


i guess this will need to be updated once those patches are merged

yep! Although with the heavy testing, Im guessing this can be left in master for both to keep testing it

mauromorales · 2023-10-03T13:44:03Z

.github/workflows/uki.yaml

+          cp tests/go.* .
+          go run github.com/onsi/ginkgo/v2/ginkgo -v --label-filter "uki" --fail-fast -r ./tests/
+      - uses: actions/upload-artifact@v3
+        if: failure()


I guess this will help debug the issue ... how does one debug from here?

gathers logs only if it fails.

mauromorales

lgtm

Itxaka commented Sep 25, 2023

View reviewed changes

Earthfile Outdated Show resolved Hide resolved

Itxaka commented Sep 25, 2023

View reviewed changes

Earthfile Outdated Show resolved Hide resolved

Itxaka commented Sep 25, 2023

View reviewed changes

framework-profile.yaml Outdated Show resolved Hide resolved

Itxaka force-pushed the uki_iso_v2 branch 2 times, most recently from 6025844 to 900a124 Compare September 29, 2023 17:03

Itxaka commented Sep 29, 2023

View reviewed changes

Itxaka force-pushed the uki_iso_v2 branch 5 times, most recently from d232029 to ec027c2 Compare October 2, 2023 14:00

Itxaka added 7 commits October 2, 2023 17:21

Improve uki iso stuff

6fee9b1

Signed-off-by: Itxaka <itxaka@kairos.io>

Up repos for fixed yip files

676f2dd

Signed-off-by: Itxaka <itxaka@kairos.io>

Just use signed efi files by default

39250c1

Signed-off-by: Itxaka <itxaka@kairos.io>

Add uki separated workflow

fc6137a

Signed-off-by: Itxaka <itxaka@kairos.io>

Improve uki tests

e90b7cf

Signed-off-by: Itxaka <itxaka@kairos.io>

use opensuse for uki tests

90faa0a

Signed-off-by: Itxaka <itxaka@kairos.io>

Itxaka force-pushed the uki_iso_v2 branch from bafc8c2 to 90faa0a Compare October 2, 2023 15:22

Itxaka requested a review from a team October 2, 2023 15:34

Itxaka marked this pull request as ready for review October 2, 2023 15:35

Itxaka and others added 4 commits October 2, 2023 21:04

Merge branch 'master' into uki_iso_v2

96a916c

Signed-off-by: Itxaka <itxaka.garcia@spectrocloud.com>

Remove detach from tests that dont need it

6564922

Signed-off-by: Itxaka <itxaka@kairos.io>

Set proper branches for uki test

d1ca04b

Signed-off-by: Itxaka <itxaka@kairos.io>

Fix yaml

4360b13

Signed-off-by: Itxaka <itxaka@kairos.io>

Itxaka force-pushed the uki_iso_v2 branch from 5d4efd9 to 4360b13 Compare October 3, 2023 09:19

mauromorales reviewed Oct 3, 2023

View reviewed changes

mauromorales approved these changes Oct 3, 2023

View reviewed changes

Itxaka merged commit 05ed541 into kairos-io:master Oct 3, 2023
22 checks passed

Itxaka deleted the uki_iso_v2 branch October 3, 2023 20:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve uki iso stuff #1854

Improve uki iso stuff #1854

Itxaka commented Sep 25, 2023 •

edited

Loading

Itxaka Sep 25, 2023

mauromorales Oct 3, 2023

Itxaka Oct 3, 2023

mauromorales Oct 3, 2023

Itxaka Oct 3, 2023

Itxaka Sep 25, 2023

mauromorales Oct 3, 2023

Itxaka Oct 3, 2023

Itxaka Sep 25, 2023

Itxaka Sep 25, 2023

Itxaka Sep 29, 2023

mauromorales Oct 3, 2023

Itxaka Oct 3, 2023

mauromorales Oct 3, 2023

Itxaka Oct 3, 2023

mauromorales left a comment

		# Use immmucore master as it has patches not released for uki
		# Use kairos-agent main branch as it has patches not released for uki

Improve uki iso stuff #1854

Improve uki iso stuff #1854

Conversation

Itxaka commented Sep 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mauromorales left a comment

Choose a reason for hiding this comment

Itxaka commented Sep 25, 2023 •

edited

Loading