Unicore bootloader causes extra reset with multi core app (IDFGH-9336) #10714

arjanmels · 2023-02-06T16:57:31Z

Answers checklist.

I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v4.4.4

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

PowerShell

Development Kit.

Custom Board

Power Supply used.

Battery

What is the expected behavior?

When booting should not trigger RTC Watchdog when using bootloader built with unicore and app with multicore.

What is the actual behavior?

On first boot it trigger RTC watchdog the first time, then the second time boto is succesfull

Steps to reproduce.

Software or hardware reset.

Debug Logs.

No response

More Information.

No response

arjanmels · 2023-02-19T10:06:15Z

I have found a workaround by overriding the app entry point and initializing the second core in the app.

I cannot call the mmu_init(1): that leads to crashes. Is this a problem, if so, what to do to be able to call this?

Overriding the entry in CmakeLists.txt with:
target_link_libraries(${COMPONENT_LIB} INTERFACE "-Wl,-eapp_entry")

and then somewhere in the app:

extern "C" void call_start_cpu0(void);

extern "C" void IRAM_ATTR app_entry(void)
{
    ESP_EARLY_LOGW(TAG, "app_entry");

#ifndef CONFIG_FREERTOS_UNICORE
    Cache_Read_Disable(1);
    Cache_Flush(1);
  
    DPORT_REG_SET_BIT(DPORT_APP_CACHE_CTRL1_REG, DPORT_APP_CACHE_MMU_IA_CLR);
//    mmu_init(1);
    DPORT_REG_CLR_BIT(DPORT_APP_CACHE_CTRL1_REG, DPORT_APP_CACHE_MMU_IA_CLR);
    
    DPORT_REG_CLR_BIT(DPORT_APP_CACHE_CTRL1_REG, DPORT_APP_CACHE_MASK_DROM0);
    
    ESP_EARLY_LOGW(TAG, "app_entry: call_start_cpu0");
#endif 

    call_start_cpu0();
}

KonstantinKondrashov · 2023-03-07T06:47:53Z

Hi @arjanmels!
Could you describe the example that you ran? and please provide sdkconfig. We need to reproduce this issue first.
What logs did you get? please share them as well.
Thanks.

arjanmels · 2023-03-07T21:56:09Z

Hi @KonstantinKondrashov,

All below is using esp-idf 4.4.4:
You can just use the empty "esp-idf/examples/getting_started/sample_project"

You need to compile it with the default sdkconfig, with only ``CONFIG_FREERTOS_UNICORE=y``` set (see attached sdkconfig_1.txt)
Program with idf.py flash (to program unicore bootloader + app)
Optional: check the app is booting properly in one go
Compile with the default sdkconfig, with # CONFIG_FREERTOS_UNICORE is not set (see attached sdkconfig_2.txt)
Program with idf.py app-flash (to only program the app, not the new bootloader)
See the app failing to boot the first time (after a hard reset), then the RTC Watchdog kick in and app successfully boots only the second time (see attached log)
boot_fail_log.txt

Best Regards,

Arjan

KonstantinKondrashov · 2023-03-08T19:32:32Z

Hi @arjanmels! Thanks for bringing up this question.
As I can say it is not an appropriate way for having different core settings for the app and bootloader. Better to have the bootloader and app with the same FREERTOS_UNICORE settings. If possible update the bootloader, otherwise need to use a workaround. Your workaround looks good, you just do those things that the bootloader should do for multicore settings.
I will be back once we made a final decision about this.

arjanmels · 2023-03-08T20:36:19Z

On my workaround, I'm a bit concerned that I cannot execute the "mmu_init(1)". Any feedback on that specifically?

I realize it is not a very common scenario and not best practise. What happened is that I initially developed for unicore (as I was not yet certain of the target chip) and that I programmed and put into the field a limited number of devices with unicore bootloader (but dualcore devices). Later I decided to start using the multicore features and that is when I encountered this problem.

Possible solutions I see:

Bootloader always initializes all cores; irrespective of unicore setting
Bootloader always unicore; second core initialization always done by the app
No change, then at least this issue documents a possible workaround.

KonstantinKondrashov · 2023-03-10T08:09:28Z

Hi @arjanmels!
I am asked about this mmu_init(1). Yes, it is ok not to call it.
mmu_init(1) is supposed to mask off all the core1 mmu entry table, but it isn't. So the core1 mmu table is wrong after mmu_init(1).

About the final solution, it has not undecided yet. Yes, definitely we will need to update the doc and probably mention this workaround, I am not sure about backporting this workaround because this fix does not work for all versions. Need to do some additional cache settings.

*) multicore app with this fix.

	app v4.4.4	app v5.0.1	app master
bootloader v4.4.4 unicore	works	works	works
bootloader v5.0.1 unicore	no	no	no
bootloader master unicore	no	no	no

arjanmels · 2023-03-10T13:36:44Z

Hi @KonstantinKondrashov,

Thank you for the confirmation on the mmu_init(1). For me the workaround is then ok for the moment (I do not plan to migrate to v5.0.1 yet).

revtronix · 2023-03-21T05:50:21Z

I would be very grateful for a workaround for v5.0.1 since I mistakenly shipped some devices with CONFIG_FREERTOS_UNICORE=y.

Here is the log output when I tried the previously mentioned workaround:

ets Jul 29 2019 12:21:46

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 271414342, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:2, clock div:2
secure boot v2 enabled
secure boot verification succeeded
load:0x3fff00b8 len:0x268
load:0x40078000 len:0x4cd8
load:0x40080400 len:0xab8
0x40080400: _init at ??:?

entry 0x4008054c
W (2418092847) app_boot: app_entry
W (2418092847) app_boot: app_entry: call_start_cpu0
I (339) cpu_start: Pro cpu up.
I (340) cpu_start: Starting app cpu, entry point is 0x4008cfb8
0x4008cfb8: call_start_cpu1 at /COMPONENT_ESP_SYSTEM_DIR/port/cpu_start.c:152

Guru Meditation Error: Core  1 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x4011064c: bad00bad bad00bad bad00bad
0x4011064c: bootloader_common_get_active_otadata at /COMPONENT_BOOTLOADER_SUPPORT_DIR/src/bootloader_common_loader.c:58

Core  1 register dump:
PC      : 0x40110652  PS      : 0x00060030  A0      : 0x80007c18  A1      : 0x3ffe7d90  
0x40110652: bootloader_init_mem at /COMPONENT_BOOTLOADER_SUPPORT_DIR/src/bootloader_mem.c:22

A2      : 0x3ffc22ac  A3      : 0x40006840  A4      : 0x00000044  A5      : 0x3ffe7e04  
A6      : 0x3ff9eae0  A7      : 0x3ff9eac9  A8      : 0x8008cfdc  A9      : 0x3ffe7d70  
A10     : 0x40000de8  A11     : 0x40000de8  A12     : 0x3ffe4358  A13     : 0x00000000  
A14     : 0x00000000  A15     : 0x00000000  SAR     : 0x00000000  EXCCAUSE: 0x00000000  
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000

KonstantinKondrashov · 2023-03-21T15:17:00Z

Hi @revtronix !
Could you try this at the top of call_start_cpu0? Does it work for you? I need some time to verify it from our side as well.

diff --git a/components/esp_system/port/cpu_start.c b/components/esp_system/port/cpu_start.c
index abd5dc8..68309c7 100644
--- a/components/esp_system/port/cpu_start.c
+++ b/components/esp_system/port/cpu_start.c
@@ -241,12 +241,24 @@ static void start_other_core(void)
 }
 #endif // !CONFIG_ESP_SYSTEM_SINGLE_CORE_MODE
 
+#include "hal/cache_ll.h"
+#include "hal/mmu_hal.h"
+#include "hal/mmu_ll.h"
+
 /*
  * We arrive here after the bootloader finished loading the program from flash. The hardware is mostly uninitialized,
  * and the app CPU is in reset. We do have a stack, so we can do the initialization in C.
  */
 void IRAM_ATTR call_start_cpu0(void)
 {
+#if 1
+    Cache_Read_Disable(1);
+    Cache_Flush(1);
+    DPORT_REG_SET_BIT(DPORT_APP_CACHE_CTRL1_REG, DPORT_APP_CACHE_MMU_IA_CLR);
+    DPORT_REG_CLR_BIT(DPORT_APP_CACHE_CTRL1_REG, DPORT_APP_CACHE_MMU_IA_CLR);
+    uint32_t bus_mask = DPORT_REG_READ(DPORT_PRO_CACHE_CTRL1_REG);
+    DPORT_REG_WRITE(DPORT_APP_CACHE_CTRL1_REG, bus_mask);
+#endif
 #if !CONFIG_ESP_SYSTEM_SINGLE_CORE_MODE
     soc_reset_reason_t rst_reas[SOC_CPU_CORES_NUM];
 #else

revtronix · 2023-03-23T02:54:06Z

Yes, I can confirm that this patch works!

Here is the startup log output:

ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x12 (SPI_FAST_FLASH_BOOT)
configsip: 271414342, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:2, clock div:2
secure boot v2 enabled
secure boot verification succeeded
load:0x3fff00b8 len:0x268
load:0x40078000 len:0x4cd8
load:0x40080400 len:0xab8
0x40080400: _init at ??:?

entry 0x4008054c
I (339) cpu_start: Pro cpu up.
I (339) cpu_start: Starting app cpu, entry point is 0x40080f38
0x40080f38: call_start_cpu1 at /COMPONENT_ESP_SYSTEM_DIR/port/cpu_start.c:147

I (0) cpu_start: App cpu up.
I (353) cpu_start: Pro cpu start user code
I (353) cpu_start: cpu freq: 240000000 Hz
I (353) cpu_start: Application information:
I (358) cpu_start: ELF file SHA256:  70dd69eec8d2f9fa...
I (364) cpu_start: ESP-IDF:          v5.1-dev-3541-g778aeae99e-dirty
I (371) cpu_start: Min chip rev:     v3.0
I (376) cpu_start: Max chip rev:     v3.99 
I (381) cpu_start: Chip rev:         v3.0
I (386) heap_init: Initializing. RAM available for dynamic allocation:
I (393) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
I (399) heap_init: At 3FFB6388 len 00001C78 (7 KiB): DRAM
I (405) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
I (411) heap_init: At 3FFD5828 len 0000A7D8 (41 KiB): DRAM
I (417) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (423) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (430) heap_init: At 4009BCD0 len 00004330 (16 KiB): IRAM
I (437) spi_flash: detected chip: gd
I (440) spi_flash: flash io: dio
I (444) flash_encrypt: Flash encryption mode is RELEASE
W (451) ADC: legacy driver is deprecated, please migrate to `esp_adc/adc_oneshot.h`
I (459) app_start: Starting scheduler on CPU0
II (n_t) kpp_tarrtd Starting[scheduler on CPU1
I (469) main_task: Calling app_main()

…gs unconditionally at startup app It makes multicore app runnable by unicore bootloader Closes espressif/esp-idf#10714

…gs unconditionally at startup app It makes multicore app runnable by unicore bootloader Closes #10714

arjanmels added the Type: Bug bugs in IDF label Feb 6, 2023

github-actions bot changed the title ~~Unicore bootloader causes extra reset with multi core app~~ Unicore bootloader causes extra reset with multi core app (IDFGH-9336) Feb 6, 2023

espressif-bot added the Status: Opened Issue is new label Feb 6, 2023

espressif-bot assigned KonstantinKondrashov Mar 3, 2023

espressif-bot added Status: In Progress Work is in progress and removed Status: Opened Issue is new labels Mar 8, 2023

espressif-bot closed this as completed in 975c138 Apr 3, 2023

espressif-bot added Resolution: Done Issue is done internally Status: Done Issue is done internally and removed Status: In Progress Work is in progress labels Apr 3, 2023

espressif-bot pushed a commit that referenced this issue May 14, 2023

esp_system: Do not rely on bootloader cache settings, do cache settin…

861a5fb

…gs unconditionally at startup app It makes multicore app runnable by unicore bootloader Closes #10714

KonstantinKondrashov mentioned this issue Jun 28, 2023

Incompatible settings for CONFIG_FREERTOS_UNICORE and CONFIG_SPIRAM (IDFGH-10359) #11617

Closed

3 tasks

espressif-bot pushed a commit that referenced this issue Sep 3, 2023

esp_system: Do not rely on bootloader cache settings, do cache settin…

1b608a1

…gs unconditionally at startup app It makes multicore app runnable by unicore bootloader Closes #10714

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unicore bootloader causes extra reset with multi core app (IDFGH-9336) #10714

Unicore bootloader causes extra reset with multi core app (IDFGH-9336) #10714

arjanmels commented Feb 6, 2023

arjanmels commented Feb 19, 2023

KonstantinKondrashov commented Mar 7, 2023

arjanmels commented Mar 7, 2023

KonstantinKondrashov commented Mar 8, 2023

arjanmels commented Mar 8, 2023

KonstantinKondrashov commented Mar 10, 2023

arjanmels commented Mar 10, 2023

revtronix commented Mar 21, 2023

KonstantinKondrashov commented Mar 21, 2023

revtronix commented Mar 23, 2023 •

edited

Unicore bootloader causes extra reset with multi core app (IDFGH-9336) #10714

Unicore bootloader causes extra reset with multi core app (IDFGH-9336) #10714

Comments

arjanmels commented Feb 6, 2023

Answers checklist.

IDF version.

Operating System used.

How did you build your project?

If you are using Windows, please specify command line type.

Development Kit.

Power Supply used.

What is the expected behavior?

What is the actual behavior?

Steps to reproduce.

Debug Logs.

More Information.

arjanmels commented Feb 19, 2023

KonstantinKondrashov commented Mar 7, 2023

arjanmels commented Mar 7, 2023

KonstantinKondrashov commented Mar 8, 2023

arjanmels commented Mar 8, 2023

KonstantinKondrashov commented Mar 10, 2023

arjanmels commented Mar 10, 2023

revtronix commented Mar 21, 2023

KonstantinKondrashov commented Mar 21, 2023

revtronix commented Mar 23, 2023 • edited

revtronix commented Mar 23, 2023 •

edited