Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute wipefs before ignition in live iso environment #947

Closed
rgl opened this issue Aug 6, 2022 · 12 comments
Closed

Execute wipefs before ignition in live iso environment #947

rgl opened this issue Aug 6, 2022 · 12 comments

Comments

@rgl
Copy link

rgl commented Aug 6, 2022

Feature Request

I need a way to wipe the boot disk before installation.

Desired Feature

I need to known how can I execute wipefs before ignition takes over the installation.

I'm asking this because, on bare metal, the disk has tainted data (e.g. another OS is there before) and I need to wipe everything to make it appear that the disk is really empty.

It seems the installer has a way to define a --pre-install install script, but I could not find a way to hook into that from the live iso environment. So, please help :-)

@bgilbert
Copy link
Contributor

bgilbert commented Aug 7, 2022

You'd do something like this:

  1. Create a script run-wipefs:
    #!/bin/bash
    wipefs -a /dev/sda
  2. Create a custom ISO image that runs the script before installing:
    coreos-installer iso customize \
        --pre-install run-wipefs \
        --dest-device /dev/sda \
        --dest-ignition your-ignition-config.ign \
        fedora-coreos-36.20220716.3.1-live.x86_64.iso -o custom.iso
  3. Boot custom.iso on the target machine.

@rgl
Copy link
Author

rgl commented Aug 7, 2022

That would work if I would create a new iso.

But how about doing that without creating a new iso? Just using kernel arguments, or even the ignition itself?

@bgilbert
Copy link
Contributor

bgilbert commented Aug 7, 2022

In general, iso customize (and pxe customize) is the recommended way to customize the install process. Is there a reason that approach won't work for you?

But yes, you can manually emulate what iso customize is doing. Boot the live image with ignition.firstboot ignition.platform.id=metal ignition.config.url=https://example.com/install.ign, where install.ign is a separate Ignition config that affects the boot of the live system. In that config, you can add a wipefs systemd service that runs After=coreos-installer-pre.target and Before=coreos-installer.service.

@rgl
Copy link
Author

rgl commented Aug 8, 2022

In general, iso customize (and pxe customize) is the recommended way to customize the install process. Is there a reason that approach won't work for you?

It requires me to take extra steps.

Its simpler to download an upstream artifact and customize it at runtime. And in this case, it seems that a pre-install hook should be easier to use at runtime.

But yes, you can manually emulate what iso customize is doing. Boot the live image with ignition.firstboot ignition.platform.id=metal ignition.config.url=https://example.com/install.ign, where install.ign is a separate Ignition config that affects the boot of the live system. In that config, you can add a wipefs systemd service that runs After=coreos-installer-pre.target and Before=coreos-installer.service.

That sounds interesting. I gave it a shot, but I'm missing something fundamental, as my sleep 9000 did not seem to prevent the installation from proceeding.

I've modified my iPXE script to include the suggested cmdline arguments (ignition.firstboot, ignition.platform.id, and ignition.config.url):

initrd ${artifacts}/initrd.img
chain --autofree --replace \
  ${artifacts}/vmlinuz \
  initrd=initrd.img \
  ignition.firstboot \
  ignition.platform.id=metal \
  ignition.config.url=${config}/live-config.ign \
  coreos.live.rootfs_url=${artifacts}/rootfs.img \
  coreos.inst.install_dev=/dev/sda \
  coreos.inst.ignition_url=${config}/config.ign

The live-config.yaml file:

variant: fcos
version: 1.4.0
systemd:
  units:
    - name: live-pre-install.service
      enabled: true
      contents: |
        [Unit]
        Description=live pre install
        After=coreos-installer-pre.target
        Before=coreos-installer.service
        [Service]
        Type=oneshot
        # TODO get the value of coreos.inst.install_dev kernel cmdline and call wipefs on it.
        ExecStart=/usr/bin/bash -c "echo XXX Hello World; sleep 9000; exit 1"
        RemainAfterExit=yes
        [Install]
        WantedBy=multi-user.target

Which was converted to live-config.ign with butane --pretty --strict live-config.yaml >live-config.ign, and resulted in:

{
  "ignition": {
    "version": "3.3.0"
  },
  "systemd": {
    "units": [
      {
        "contents": "[Unit]\nDescription=live pre install\nAfter=coreos-installer-pre.target\nBefore=coreos-installer.service\n[Service]\nType=oneshot\n# TODO get the value of coreos.inst.install_dev kernel cmdline and call wipefs on it.\nExecStart=/bin/bash -c \"echo XXX Hello World; sleep 9000; exit 1\"\nRemainAfterExit=yes\n[Install]\nWantedBy=multi-user.target\n",
        "enabled": true,
        "name": "live-pre-install.service"
      }
    ]
  }
}

Can you please help me understand what is missing?

@bgilbert
Copy link
Contributor

bgilbert commented Aug 8, 2022

Oh, you're using PXE boot and not the ISO; that makes more sense. Users typically don't want to hand-type a bunch of kernel arguments at the ISO boot prompt.

Your unit isn't starting because we don't activate multi-user.target if coreos.inst.install_dev is specified on the kernel command line. I should have said that the Install section should use RequiredBy=coreos-installer.service. (The installation should fail if wipefs does, so WantedBy isn't strong enough.)

@rgl
Copy link
Author

rgl commented Aug 8, 2022

Oh, you're using PXE boot and not the ISO; that makes more sense. Users typically don't want to hand-type a bunch of kernel arguments at the ISO boot prompt.

Oh, I wanted to write live pxe instead of live iso. Sorry!

Your unit isn't starting because we don't activate multi-user.target if coreos.inst.install_dev is specified on the kernel command line. I should have said that the Install section should use RequiredBy=coreos-installer.service. (The installation should fail if wipefs does, so WantedBy isn't strong enough.)

Thank You! Now its working nicely! And I now understand a bit more about systemd. And indeed, it aborts the boot when the script fails.

I need further help thou :-)

I need to hook into the post-install phase somehow. I have to create the uefi boot option after the installation is complete successfully. Can you please hint me in what values should I use in the .service file?

@bgilbert
Copy link
Contributor

bgilbert commented Aug 8, 2022

For post-install, you'll want After=coreos-installer.service, Before=coreos-installer.target, and RequiredBy=coreos-installer.target.

@rgl
Copy link
Author

rgl commented Aug 8, 2022

Thank You! It worked nicely! :-)

For future reference, here the full configuration, which wipes the boot disk before installation, then resets the uefi boot options after installation.

iPXE partial script:

initrd ${artifacts}/initrd.img
chain --autofree --replace \
  ${artifacts}/vmlinuz \
  initrd=initrd.img \
  ignition.firstboot \
  ignition.platform.id=metal \
  ignition.config.url=${config}/install.ign \
  coreos.live.rootfs_url=${artifacts}/rootfs.img \
  coreos.inst.install_dev=/dev/sda \
  coreos.inst.ignition_url=${config}/config.ign

install.yaml ignition source:

variant: fcos
version: 1.4.0
storage:
  files:
    - path: /usr/local/bin/pre-install.sh
      mode: 0o555
      contents:
        inline: |
          #!/usr/bin/bash
          set -euxo pipefail
          boot_device="$(sed -nE 's,.+ coreos\.inst\.install_dev=([^ ]+).*,\1,p' /proc/cmdline)"
          # wipe the boot disk.
          wipefs --all $boot_device
          blkdiscard $boot_device
    - path: /usr/local/bin/post-install.sh
      mode: 0o555
      contents:
        inline: |
          #!/usr/bin/bash
          set -euxo pipefail
          boot_device="$(sed -nE 's,.+ coreos\.inst\.install_dev=([^ ]+).*,\1,p' /proc/cmdline)"
          firmware="$([ -d /sys/firmware/efi ] && echo 'uefi' || echo 'bios')"
          if [ "$firmware" == 'uefi' ]; then
            # remove all the boot options.
            efibootmgr \
              | sed -nE 's,^Boot([0-9A-F]{4}).*,\1,gp' \
              | xargs -I% efibootmgr --quiet --delete-bootnum --bootnum %
            # create the fedora boot option.
            # NB if we do not set any boot option, the firmware will recover/discover
            #    the boot options at the next boot. unfortunately, in my test HP
            #    EliteDesk 800 35W G2 Desktop Mini, this requires an extra reboot,
            #    which messes with the ethernet speed by switching it to 10 Mbps, so
            #    we also create the boot option.
            efibootmgr \
              -c \
              -d "$boot_device" \
              -p 2 \
              -L Fedora \
              -l '\EFI\fedora\shimx64.efi'
          fi
systemd:
  units:
    - name: live-pre-install.service
      enabled: true
      contents: |
        [Unit]
        Description=live pre install
        After=coreos-installer-pre.target
        Before=coreos-installer.service
        [Service]
        Type=oneshot
        ExecStart=/usr/local/bin/pre-install.sh
        RemainAfterExit=yes
        [Install]
        RequiredBy=coreos-installer.service
    - name: live-post-install.service
      enabled: true
      contents: |
        [Unit]
        Description=live post install
        After=coreos-installer.service
        Before=coreos-installer.target
        [Service]
        Type=oneshot
        ExecStart=/usr/local/bin/post-install.sh
        RemainAfterExit=yes
        [Install]
        RequiredBy=coreos-installer.target

@rgl rgl closed this as completed Aug 8, 2022
@bgilbert
Copy link
Contributor

bgilbert commented Aug 8, 2022

       # NB if we do not set any boot option, the firmware will recover/discover
       #    the boot options at the next boot. unfortunately, in my test HP
       #    EliteDesk 800 35W G2 Desktop Mini, this requires an extra reboot,

It requires an extra reboot if a TPM is present and enabled, since the presence of fallback.efi in the boot chain would break the PCR values. Eventually we plan to stop setting UEFI boot variables altogether; see coreos/fedora-coreos-tracker#946.

@rgl
Copy link
Author

rgl commented Aug 9, 2022

Interesting. That seems to be a different problem that I'm yet to encounter.

Here I have to create the uefi boot option because the extra reboot messes with the network interface speed (because this machine uses AMT for remote management, and when Remote Desktop is active, it changes the speed to a crawling 10 Mbps).

After I create that uefi boot option, there is no extra reboots, the system boots directly to grub/fedora, without any extra reboots.

But, I'm still missing the TPM reset part. Maybe when I do that, I'll get into the problem you've linked. I'll keep an eye on it.

Thanks for the heads up!

@bgilbert
Copy link
Contributor

I meant that the extra reboot you're seeing is happening because you have a TPM. You can avoid it either by manually adding a boot entry, as you're doing, or by disabling the TPM.

@rgl
Copy link
Author

rgl commented Aug 10, 2022

Oh, I misunderstood what was happening. Its not the firmware that is doing the reboot, its the shim (which is shared across distros). Thanks for clearing that up :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants