OKD on SCOS (4.14 2024-01-30) deployment fails on vSphere UPI, SCOS bad shim signature

[kai-uwe-rommel]

Describe the bug
Tried to deploy a okd-scos-2024-01-30-032525 cluster on vSphere UPI.
After pulling the SCOS image onto the bootstrap VM with FCOS, the reboot fails with "bad shim signature"

Version
okd-scos-2024-01-30-032525
on vSphere UPI

How reproducible
100%

Log bundle
No log bundle at this time.

[bdlink]

Be sure you are not using secure boot for the guest machine.

Under Settings, click the VM Options tab, and expand Boot Options. I am using Bios for firmware, which has worked in the past. If you are using EFI firmware, be sure to deselect the Secure Boot check box.

[LorbusChris]

Broken Secure Boot on SCOS is a known issue: https://issues.redhat.com/browse/RHEL-4391

[kai-uwe-rommel]

Yes, disabling secure boot as a workaround solved the problem. I could successfully deploy such an OKD/SCOS cluster.
I guess we need to wait for an upstream fix in CentOS Stream.
Also there already seems to be an OCPBUGS issue for my problem: https://issues.redhat.com/browse/OCPBUGS-21838

[kai-uwe-rommel]

If someone is looking for a workaround when installing on vSphere UPI ...

I wrote our automation in PowerShell to be able to use PowerCLI for VMware vSphere tasks. What I did there to disable Secure Boot:

	$vm = New-Vm -Name $vmName -VMHost $vmHost -Template $vmTemplate -Datastore $vmDatastore -Location $vmFolder -Confirm:$false

    ......
    $vmSecureBoot = $false # in case of SCOS ...

	$secureBootStatus = $vm.ExtensionData.Config.BootOptions.EfiSecureBootEnabled
	If ($secureBootStatus -ne $vmSecureBoot) {
	    $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
	    $spec.BootOptions = New-Object VMware.Vim.VirtualMachineBootOptions
	    $spec.BootOptions.EfiSecureBootEnabled = $vmSecureBoot
	    $taskMoRef = $vm.ExtensionData.ReconfigVM_Task($spec)
	    Get-Task -Id $taskMoRef | Wait-Task | Out-Null
	}
   

[JaimeMagiera]

I've grabbed your tip and will put it in the release notes. Closing this for completeness.

[kai-uwe-rommel]

BTW, FCOS 40 also has this "bad shim signature" problem now.

[jcpowermac]

@kai-uwe-rommel openshift/installer#8793

But probably only will be backported to 4.16

edit: (oops I misread this was upi, the change above was for ipi)