Change EFI-System partition format from fat16 to fat32

[buckaroogeek]

Bug

Raspberry Pi 4 will not boot Fedora CoreOS if the EFI-System partition is FAT16. Convert to FAT32 and the RPi4 boots. The coreos-installer appears to use FAT16 for the EFI system partition. I could not see an option to change this at coreos-installer run time.

Host Operating System Version

Fedora 34

Target Operating System Version

Fedora CoreOS 34

coreos-installer Version

coreos-installer-0.10.0-2.fc34.x86_64

Expected Behavior

Use coreos-installer to create a bootable USB mass storage device for Raspberry Pi 4. CoreOS should boot once coreos-installer completes without error and the USB device is plugged into the RPi4 and powered on.

Actual Behavior

FCOS will not boot on the RPi4 until the EFI-System partition is reformatted as FAT32. It appears that coreos-installer formats the partition as FAT16.
The RPi4 displays a message: Trying partition 0. Trying to read partition as fat. (with other text), This message block repeats blocking the boot process.

Reproduction Steps

1. Install FCOS to usb (e.g. sudo coreos-installer install /dev/sdh -i ./fc1.ign -f fedora-coreos-34.20210919.3.0-etal.aarch64.raw --offline --insecure)
2. Dismount USB from Fedora workstation
3. Mount USB on RPi4 (usb3 port).
4. Power on.
5. RPi4 cannot read partition.

Other Information

I am using v 1.31 of the pftp/uefi firmware. I can manually reformat the partition to FAT32 and FCOS boots as expected.

[jlebon]

As far as I can tell from the UEFI specification, the firmware is supposed to support FAT16 or FAT32:

13.3.1.1 File System Format

The EFI firmware must support the FAT32, FAT16, and FAT12 variants of the EFI file system. What variant
of EFI FAT to use is defined by the size of the media. The rules defining the relationship between media
size and FAT variants is defined in the specification for the EFI file system.

We seem to not be specifying the actual FAT size when running mkfs.fat, which looks like it makes it also auto-detect the right one based on size:

-F FAT-SIZE
           Specifies  the type of file allocation tables used (12, 16 or 32 bit).  If nothing is speci‐
           fied, mkfs.fat will automatically select between 12, 16 and 32 bit, whatever fits better for
           the filesystem size.

which seems to fit with the UEFI definition.

AFAIK, we've had other users successfully boot FCOS on the Raspberry Pi 4. I suspect this may be related to the fact that you're booting via USB instead of SD card? @dustymabe Have you tried booting via USB?

[bgilbert]

My understanding is that the ESP on a non-removable disk must be FAT32. UEFI spec 2.9 section 13.3 says:

EFI encompasses the use of FAT32 for a system partition, and FAT12 or FAT16 for removable media.

We had the same bug in Container Linux (coreos/bugs#2246).

[buckaroogeek]

Thanks for the information about the specification and updates. I should spell check before I submit! I recall that during my initial efforts to install and run FCOS on RPi4 hardware last February (see https://forums.raspberrypi.com/viewtopic.php?t=304318 for a write up if interested), this was not an issue I ran into. The USB disk I uses is a KINGSTON SA400S37120G (120 GB). Please let me know if I can provide any more information and/or testing.

[dustymabe]

@dustymabe Have you tried booting via USB?

I've been recently testing things via both uboot and UEFI (working on documentation to add to our docs site). I have to admit I have seen some cases where the pi4 will loop on trying partition 0 like @buckaroogeek suggested, but I've also taken that exact USB out and plugged it back in and it worked the next time, which was odd.

Maybe use of FAT16 makes the issue more likely to happen? Maybe I'm misremembering? I definitely know things are working on an SD card just fine the way it is today.

[buckaroogeek]

I checked the eeprom firmware page on github (https://github.com/raspberrypi/rpi-eeprom/blob/master/firmware/release-notes.md) and see this note: "Update the FAT detection to support FAT16 for EFI/ESD paritions with GPT instead of assuming FAT32. The latest firmware is also required for a similar update." This is for a 4 Oct update.

[bgilbert]

This is a corner case that likely affects more than just the Pi, so we should fix it regardless. Fix in coreos/coreos-assembler#2491.

[bgilbert]

And thanks for the report!

[buckaroogeek]

Glad to help.

[dustymabe]

Going to re-open until the change lands