qemu: Using `-bios` for firmware does not work with OVMF firmware

[c3d]

Get your issue reviewed faster

From code inspection, this applies to both the 1.x and 2.0 branches equally.

Description of problem

When using the firmware configuration option, the runtime translates that into a -bios command-line option for qemu. This will not work with an OVMF firmware, which also needs separate OVMF_CODE.fd for the read-only portion of the firmware and OVMF_VARS.fd for the associated variables.

Expected result

For an OVMF firmware, you'd expect the qemu command line to contain two entries, one for the read-only section of the firmware (OVMF_CODE.fd) and one for the read-write portion (OVMF_VARS.fd). This is typically done using:

 -drive if=pflash,format=raw,readonly,file=/path/to/OVMF_CODE.fd \
 -drive if=pflash,format=raw,file=/path/to/OVMF_VARS.fd \

See fuller story here, for example: https://wiki.debian.org/SecureBoot/VirtualMachine

Actual result

The current code generates the simpler -bios option that looks like:

-bios /path/to/OVMF_CODE.fd

In that case, qemu will most likely not know how to find the related OVMF_VARS.fd. This may or may not be important.

Further information

Example of command-line option with an OVMF firmware:

-name sandbox-csh -uuid a9ecf610-ebb5-42b4-bd8c-d72445f60601 -machine q35,accel=kvm,kernel_irqchip -cpu host,pmu=off -qmp unix:/run/vc/vm/csh/qmp.sock,server,nowait -m 2048M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile= -device virtio-serial-pci,disable-modern=false,id=serial0,romfile= -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/csh/console.sock,server,nowait -device virtio-scsi-pci,id=scsi0,disable-modern=false,romfile= -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0,romfile= -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-3113278101,guest-cid=3113278101,romfile= -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared,romfile= -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/csh/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=2048M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinuz-sev-es-5.6-v3.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 quiet panic=1 nr_cpus=240 scsi_mod.scan=none root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -bios /usr/local/share/qemu/OVMF_CODE.fd.kata -pidfile /run/vc/vm/csh/pid -smp 1,cores=1,threads=1,sockets=240,maxcpus=240

[c3d]

Some additional digging led me to believe that the -bios option actually may work well enough for our use case. Specifically, I tried the following manually:

 qemu-kvm -m 2048 -M q35,accel=kvm,kernel_irqchip  -bios /usr/share/OVMF/OVMF_CODE.fd -kernel /boot/vmlinuz -initrd /tmp/kata-test/initrd.img

and I got the kernel to boot reasonably far. This is not with secure boot enabled, though, so this may not work as intended if what are looking for is SecureBoot. But at least, qemu is smart enough to "do the right thing" when receiving both the -kernel and -bios options.

[c3d]

Setting "low priority" label since this seems to work somewhat OK with the current -bios option.

[c3d]

I was wrong about what the code generates. These seems to be a difference between 1.x and 2.0. According to a discussion with Sandeep Gupta on kata-dev, we have a -bios option for 2.0:

 -bios /usr/local/share/qemu/OVMF_CODE.fd.kata

But a partially correct -drive if=pflash option for 1.x:

 -drive if=pflash,format=raw,unit=0,file=/usr/local/share/qemu/OVMF_CODE.fd.kata,readonly

It's not very clear where that comes from in the govmm code:

func (config *Config) appendBios() {
	if config.Bios != "" {
		config.qemuParams = append(config.qemuParams, "-bios")
		config.qemuParams = append(config.qemuParams, config.Bios)
	}
}

I see nothing there that could produce the -drive option.

[jodh-intel]

/cc @devimc, @sboeuf, @markdryan.

[c3d]

See thread discussing the topic.

Answer from Laszlo Ersek:

"-bios" should never be used with OVMF. If you don't need non-volatile
UEFI variables to persist from one boot of the domain to the next,
that's fine; in that case, please use a throw-away regular file for the
variable store pflash chip. this file should be instantiated from
"OVMF_VARS.fd", if you don't want the Secure Boot operational mode
enabled right off the bat; otherwise, please copy (create) the domain's
varstore from "OVMF_VARS.secboot.fd", before launching QEMU.

We have never tested SEV with "-bios" (well, because "-bios" is wrong to
use with OVMF in the first place, regardless of SEV); and yes there have
been SEV-related changes, specific to the flash driver in OVMF. (QEMU
too received SEV-related changes around the parts responsible for
loading OVMF, in "hw/i386/pc_sysfw.c".)

The particulars of a SEV failure, when using "-bios", aren't even worth
investigating -- please try to reproduce the issue first with two pflash
chips.

So pretty clear that we need to change the qemu command-line if we want to support safeboot and SEV.

[jimcadden]

Ah, so I hit this problem in my SEV patches and worked around it in the follow way: In govmm I create the -drive if=pflash ... for OVMF at the same time as the -object sev-guest .... In kata-runtime, I specify the path to OVMF_CODE.fd in the config using firmware=, same as Bios, but I clear qemuConfig.Bios="" prior to govmm so that a -bios is never appended.

Not the most elegant, but it works with few code changes.

But, no support for OVMF_VARS.fd in this work-around solution.

[zvonkok]

Do we have any progress on this? Isn't this a requirement for confidential containers to load OVMF firmware? The other features that rely on this working are secure boot or measured boot. Running with the latest kata code the -bios is still used for the OVMF.

@jodh-intel

[zvonkok]

We have good support for OVMF, we can close this.