Add rpi-eeprom firmware #939

tmm1 · 2020-10-27T01:07:19Z

This will allow us to take advantage of the RPI4 self-update feature to push new stable eeprom bootloader firmwares as part of HassOS updates

cc #899 (comment) @Jpsy

pvizeli

Nice work

buildroot-external/configs/rpi4_64_defconfig

agners · 2020-10-27T09:49:49Z

Seems to work here, with enable_uart=1 I can see that the EEPROM is being updated:

Read config.txt bytes     1826 hnd 0x00000780 hash '1850978bb5bda2bb'
Read pieeprom.upd bytes   524288 hnd 0x00000136 hash '5159f01d9c15fe32'
Reading EEPROM: 524288
Writing EEPROM
+++++++++++++++++++++++++.......++++++++++++++++++++++++++++++++++++++++++++++++++..............................................
Verify BOOT EEPROM
Reading EEPROM: 524288

pvizeli · 2020-10-27T09:51:35Z

On the next OTA update, the file is present again, do that flash it again?

agners · 2020-10-27T10:06:30Z

The files are still there after the update, so I assume it detects if its newer? I don't have diagnostic output on UART anymore since the update also sets the config back...

pvizeli · 2020-10-27T10:09:35Z

Look like, so we have still open the rpi4_devconfig before we can merge it

agners · 2020-10-27T15:20:50Z

I tried to make it update again, but it just wouldn't... I am now on 2020-09-03, and it seems that this version does not automatically update from SD card. Quote from https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2711_bootloader_config.md:

Self-update is not enabled in SD boot; the ROM can already load recovery.bin from the SD card.

tmm1 · 2020-10-27T15:33:30Z

The files are still there after the update, so I assume it detects if its newer?

Yes it checks and only applies the .upd file is the current firmware does not match. It will show this info over HDMI as well telling you if update was skipped.

I don't have diagnostic output on UART anymore since the update also sets the config back...

This part is annoying for development. If you have a custom eeprom with NETCONSOLE or UART it gets overwritten. Maybe we can have a special dev file inside the boot partition and if it exists the eeprom file is removed or renamed during rauc update?

Self-update is not enabled in SD boot; the ROM can already load recovery.bin from the SD card.

Correct, there was some big problem with self-update from SD card: raspberrypi/rpi-eeprom#209

So this technique is mostly useful for USB boot scenarios. In this PR I customized USB_PWR_OFF_TIME for the eeprom to fix issues with some types of drives: raspberrypi/rpi-eeprom#241

tmm1 · 2020-10-27T15:34:52Z

buildroot-external/package/rpi-eeprom/rpi-eeprom.mk

+
+define RPI_EEPROM_BUILD_CMDS
+	$(@D)/rpi-eeprom-config $(@D)/$(RPI_EEPROM_FIRMWARE_PATH) --out $(@D)/default.conf
+	(cat $(@D)/default.conf | grep -v ^$$; echo USB_MSD_PWR_OFF_TIME=1500) > $(@D)/boot.conf


Here I changed USB_MSD_PWR_OFF_TIME from 1000 to 1500 as workaround for some drives not booting after soft-reboot.

Soon there will be a new beta firmware with a better fix, so we can use this same technique to update to that as part of HassOS update.

agners · 2020-10-27T16:04:29Z

The files are still there after the update, so I assume it detects if its newer?

Yes it checks and only applies the .upd file is the current firmware does not match. It will show this info over HDMI as well telling you if update was skipped.

Not in the SD card case, it seems.

I don't have diagnostic output on UART anymore since the update also sets the config back...

This part is annoying for development. If you have a custom eeprom with NETCONSOLE or UART it gets overwritten. Maybe we can have a special dev file inside the boot partition and if it exists the eeprom file is removed or renamed during rauc update?

I tried to just rebuild the upd file manually and flash it. I think its good enough, I hope we don't have to do this very often 😄

Self-update is not enabled in SD boot; the ROM can already load recovery.bin from the SD card.

Correct, there was some big problem with self-update from SD card: raspberrypi/rpi-eeprom#209

So this technique is mostly useful for USB boot scenarios. In this PR I customized USB_PWR_OFF_TIME for the eeprom to fix issues with some types of drives: raspberrypi/rpi-eeprom#241

Hm, yeah that is a bit annoying. I think most people still run SD-card boot. Can we update via recovery.bin?
I tried that, but I am not sure where to get recovery.bin from... Is that just the same file with different name? That I tried, it did not pick it up.

pvizeli · 2020-10-27T16:57:31Z

This part is annoying for development. If you have a custom eeprom with NETCONSOLE or UART it gets overwritten. Maybe we can have a special dev file inside the boot partition and if it exists the eeprom file is removed or renamed during rauc update?

We should not allow user to customize it. It's already hard to maintain, if every device has x variation to run, it's killing us or bite later in our a**. We decide what we want that people run and we can support.

tmm1 · 2020-10-27T17:51:23Z

I think most people still run SD-card boot. Can we update via recovery.bin?

Yes I think so. The recovery.bin is a special firmware, but once it loads it will look for pieeprom.upd and flash it, then rename itself to recovery.000

You can find it here: https://github.com/raspberrypi/rpi-eeprom/blob/master/firmware/stable/recovery.bin

According to https://www.raspberrypi.org/documentation/hardware/raspberrypi/booteeprom.md, the recovery.bin can only be loaded from SD and is ignored on USB boot.

So I think we simply need to place the recovery.bin into boot and then it will pick up and flash the pieeprom.upd on the next boot.

If the bootloader update image is called pieeprom.upd then recovery.bin renames itself to recovery.000 and resets the CPU. Since recovery.bin is no longer present the ROM loads the newly updated bootloader from SPI EEPROM and the OS is booted as normal.

agners · 2020-10-27T19:17:41Z

You can find it here: https://github.com/raspberrypi/rpi-eeprom/blob/master/firmware/stable/recovery.bin

This does not work for me either. Leads to a red screen 😕

this improves boot from USB and speeds up boot times. also includes sd card v1 boot reliability. see raspberrypi/rpi-eeprom#246 i removed the previous USB_MSD_PWR_OFF_TIME override since it is no longer required. added HDMI_DELAY=0 so hdmi display is always visible for debugging.

agners · 2020-11-24T18:57:46Z

I am generally ok merging this, however currently it uses a beta firmware. I rather prefer not shipping a beta firmware. Let's wait until the firmware gets promoted to stable.

Have there been issues with 2020-09-03?

tmm1 · 2020-11-24T19:07:34Z

With the 2020-09 stable firmware, many drives get stuck inside the bootloader after a soft reboot.

raspberrypi/rpi-eeprom#241

agners · 2020-11-25T23:32:20Z

Interesting thread. Thinking a bit more about this I think we should add the current beta update as you propose: Only people with USB boot will get the update applied. And for those this new beta firmware seems to really bring improvements. And since your tests were successful too, let's give it a go.

Jpsy · 2020-11-26T01:55:01Z

Would this mean that starting with 5.6 there will be no more need to update the RPI boot firmware through Raspberry Pi Imager before installing HA on an RPi 4 with SSD attached through USB?

Or will it still be necessary to update with the Imager at least once to get HA running for the first time. And from there on HA would install further boot EEPROM updates?

I would update the guide to reflect your answer.

pieeprom.upd/sig files which we can put into the bootfs partition of the rpi4 image so that a rpi4 (if booted from USB) will be able to update its EEPROM immediately. This refs home-assistant/operating-system#939

agners · 2020-12-15T07:51:56Z

Would this mean that starting with 5.6 there will be no more need to update the RPI boot firmware through Raspberry Pi Imager before installing HA on an RPi 4 with SSD attached through USB?

There is a chicken egg issue here: The update can only happen if the current firmware supports that feature/is capable to actually talk to that USB SSD drive attached. This is not the case with older firmwares. So depending on what is on the Pi now, a manual upgrade is required first still...

add rpi-eeprom

fa5d562

homeassistant added the cla-signed label Oct 27, 2020

pvizeli approved these changes Oct 27, 2020

View reviewed changes

pvizeli reviewed Oct 27, 2020

View reviewed changes

buildroot-external/configs/rpi4_64_defconfig Show resolved Hide resolved

tmm1 commented Oct 27, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

frenck added the hacktoberfest-accepted label Oct 31, 2020

tmm1 added 3 commits November 3, 2020 17:01

on 32-bit builds too

28c9f96

expand info

3ca1fb6

agners merged commit 1fa53bc into home-assistant:dev Nov 25, 2020

agners added the board/raspberrypi Raspberry Pi Boards label Nov 25, 2020

JMan7777 mentioned this pull request Dec 1, 2020

hassio.host_reboot not rebooting host (Raspberry Pi 4B 8GB). #888

Closed

snippem mentioned this pull request Dec 2, 2020

Has os 5.6 only only boots by unplug and plugin SSD #1045

Closed

7 tasks

agners mentioned this pull request Dec 11, 2020

Hassos 5.7 only boots from SDD on USB3 via unpowered USB3 HUB #1079

Closed

7 tasks

This was referenced Dec 16, 2020

SOLVED: SSD Enclosures with Jmicron chips JMS580 and JMS583 don't boot up. RPi4. raspberrypi/rpi-eeprom#266

Closed

RPi4 - not upgrading to 5.8 from 4.17 #1091

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add rpi-eeprom firmware #939

Add rpi-eeprom firmware #939

tmm1 commented Oct 27, 2020

pvizeli left a comment

agners commented Oct 27, 2020

pvizeli commented Oct 27, 2020

agners commented Oct 27, 2020

pvizeli commented Oct 27, 2020

agners commented Oct 27, 2020

tmm1 commented Oct 27, 2020

tmm1 Oct 27, 2020

agners commented Oct 27, 2020

pvizeli commented Oct 27, 2020 •

edited

tmm1 commented Oct 27, 2020 •

edited

agners commented Oct 27, 2020

This comment has been minimized.

agners commented Nov 24, 2020

tmm1 commented Nov 24, 2020

agners commented Nov 25, 2020 •

edited

Jpsy commented Nov 26, 2020 •

edited

agners commented Dec 15, 2020

Add rpi-eeprom firmware #939

Add rpi-eeprom firmware #939

Conversation

tmm1 commented Oct 27, 2020

pvizeli left a comment

Choose a reason for hiding this comment

agners commented Oct 27, 2020

pvizeli commented Oct 27, 2020

agners commented Oct 27, 2020

pvizeli commented Oct 27, 2020

agners commented Oct 27, 2020

tmm1 commented Oct 27, 2020

tmm1 Oct 27, 2020

Choose a reason for hiding this comment

agners commented Oct 27, 2020

pvizeli commented Oct 27, 2020 • edited

tmm1 commented Oct 27, 2020 • edited

agners commented Oct 27, 2020

This comment has been minimized.

agners commented Nov 24, 2020

tmm1 commented Nov 24, 2020

agners commented Nov 25, 2020 • edited

Jpsy commented Nov 26, 2020 • edited

agners commented Dec 15, 2020

pvizeli commented Oct 27, 2020 •

edited

tmm1 commented Oct 27, 2020 •

edited

agners commented Nov 25, 2020 •

edited

Jpsy commented Nov 26, 2020 •

edited