2-Stage Application Concept (IDFGH-8718)

[franz-ms-muc]

Is your feature request related to a problem?

not Really.

Describe the solution you'd like.

We want to extend the Application Bootup.

now it is like https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/startup.html.

1. First stage bootloader
2. Second stage bootloader
3. Application startup executes

Our goal is to have another way to Flash the OTA0 and OTA1 Partitions, over a so-Called FBL (Flash Bootloader) which is able to get a new Application Binary over CAN-BUS (J1939, UDS). 

Therefore our intended Startup Flow would be:

1. First stage bootloader
2. Second stage bootloader
3. FBL startup executes, and
   1. either flash OTA0 or OTA1 Partitions (initiated by a Magic CAN-Packet)
   2. or decide to go on with 4. 
4. Application startup executes

i think @igrr is a guy who can answer this. 

Describe alternatives you've considered.

Alternative considered is to boot FBL, and then make a Reset, 

so the Second stage bootloader sees "we are in a warmstart" and start OTA1 or OTA2. But then we must extend the Bootloader, and it must know that FBL has already run, so decide if it is a warmstart or a coldstart.

Additional context.

No response

[MartinPatarinski]

Here is also a diagram of the startup flow , that is described above

[igrr]

I think something similar has been implemented in a number of projects already, as follows:

• "Factory" partition is used as the "3rd stage" bootloader, which can download applications from various sources (Wi-Fi, SD card, etc)
• "OTA" partitions are used to store the actual applications

This enables the following workflows:

• Normal boot: 2nd stage bootloader loads the app from one of the OTA slots, the app starts running.
• Upgrade:
  • OTA app receives an event that triggers the upgrade.
  • It calls esp_ota_set_boot_partition(factory_partition) and restarts.
  • The 2nd stage bootloader loads the factory app because it is now set as the boot partition.
  • The factory app downloads the update from one of the interfaces and flashes it into an OTA slot.
  • The factory app calls esp_ota_set_boot_partition(ota_partition) and restarts.
  • The 2nd stage bootloader loads the app from the selected OTA partition.
• Fallback:
  • If an OTA app fails to boot after the upgrade, the 2nd stage bootloader can switch the default boot app to "factory", enabling recovery process. This can be done using CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE option.

For example, in one such project, the "factory" app would also present a menu on an LCD screen, letting the user to select the app ("game") to boot. The "factory" app also set up a webserver over Wi-Fi, letting users to update other apps ("games"). In your example, the "factory" app can get the update over CAN bus.

[franz-ms-muc]

Von: Ivan Grokhotkov ***@***.***> Gesendet: Freitag, 11. November 2022 11:35 An: espressif/esp-idf ***@***.***> Cc: Franz Höpfinger ***@***.***>; Author ***@***.***> Betreff: Re: [espressif/esp-idf] 2-Stage Application Concept (IDFGH-8718) (Issue #10159) I think something similar has been implemented in a number of projects already, as follows: * "Factory" partition is used as the "3rd stage" bootloader, which can download applications from various sources (Wi-Fi, SD card, etc) * "OTA" partitions are used to store the actual applications This enables the following workflows: * Normal boot: 2nd stage bootloader loads the app from one of the OTA slots, the app starts running. * Upgrade: * OTA app receives an event that triggers the upgrade. [FH] @***@***.******@***.***> this would mean: we integrate the Detection of the „Magic Packet“ into the Main App. So the Update can be started anytime, no need to flood the Bus with the Magic Packet. The Box can Boot, run 1 hour, then the Magic Packet is received by the normal App, and the following happens * It calls esp_ota_set_boot_partition(factory_partition) and restarts. [FH] this is done inside the app. * The 2nd stage bootloader loads the factory app because it is now set as the boot partition. * The factory app downloads the update from one of the interfaces and flashes it into an OTA slot. * The factory app calls esp_ota_set_boot_partition(ota_partition) and restarts. [FH] this could even be happening in case of Error, Timeout, no Binary received etc.. * The 2nd stage bootloader loads the app from the selected OTA partition. * Fallback: * If an OTA app fails to boot after the upgrade, the 2nd stage bootloader can switch the default boot app to "factory", enabling recovery process. This can be done using CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE option. For example, in one such project, the "factory" app would also present a menu on an LCD screen, letting the user to select the app ("game") to boot. The "factory" app also set up a webserver over Wi-Fi, letting users to update other apps ("games"). In your example, the "factory" app can get the update over CAN bus. [FH] yes, really cool concept. If both apps OTA1 and OTA2 are damaged: the Factory App allows reprogramming. If the App is running, but the Part where the magic packet is received and the switch to the factory app is done is damaged, then we have a Problem. This can be avoided by good Testing, and maybe inventing a 2nd way of getting into Factory, e.g. over Wifi or MQTT. Cool concept!! — Reply to this email directly, view it on GitHub<#10159 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQSZUH4ZVVX4ZNJFL4Q6BNLWHYOO3ANCNFSM6AAAAAAR5M5CLI>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>> [https://www.meisterschulen-mchn.de/images/meisterschulen/allgemein/4-logos/logofc.jpg] Meisterschulen am Ostbahnhof Mühldorfstr. 6 81671 München Tel.: 089 416002 - 0 Fax: 089 416002 - 111 Internet: www.meisterschulen-münchen.de<http://www.meisterschulen-mchn.de>

[MartinPatarinski]

Yes, we have thought about putting the FBL as Factory, but it will not be possible to update the FBL after that.
Normally, we want also to add a so called "Bootloader updater" or "Bootkiller", which is downloaded in OTA partitions and updates the FBL itself.
If we choose to use factory we would not be able to use the OTA APIs to update the FBL, cause it is factory and not writable.
Maybe it is possible to write it directly via SPI Flash driver, but would the SSBL be able to read the updated FBL/Factory correctly. If there is a header check, this would make the SSBL consider the new image as invalid.

[igrr]

cause it is factory and not writable.

There's actually no restriction on writing to 'factory' partition. I think this scheme can be extended to make the factory partition updatable. You can add another partition (ota2) which would contain the dedicated 'factory partition updater app'. If the main app gets an event that signals the factory partition update, it sets the boot app to ota2 and resets. The rest of the logic is similar.

You may use a customized 2nd stage bootloader to implement custom boot priorities, if necessary — please refer to custom bootloader examples for that.

[MartinPatarinski]

We have tested that if CONFIG_SPI_FLASH_DANGEROUS_WRITE_ALLOWED is enabled in the menuconfig, then it is possible to write the factory partition via the SPI flash driver. Also we have created custom boot priorities and this way our bootloader FBL is started every time after power on and checks for reprogramming request. If there is no such, it restarts and then SSBL starts the selected OTA partition that contains the main application.

[diplfranzhoepfinger]

@igrr is this CONFIG_SPI_FLASH_DANGEROUS_WRITE_ALLOWED neccessary ? 

du to my understanding it is protecting Partition Table and Bootloader, not the Factory App right ?

[igrr]

CONFIG_SPI_FLASH_DANGEROUS_WRITE_ALLOWED is only necessary to overwrite:

• the bootloader
• the partition table
• the partition from which the application is currently running from.

So if you aren't trying to overwrite factory partition while running the app from the factory partition, this option should not be necessary.

[igrr]

@franz-ms-muc it seems that the question has been answered, can we close the issue now?

[franz-ms-muc]

Close

[franz-ms-muc]

@igrr can you give a Hint to https://www.esp32.com/viewtopic.php?f=13&t=31895&sid=7e8498b82177cfeb323f1ff006b10162

Thanks !

[igrr]

Have replied.

[franz-ms-muc]

Outstanding good Support !