V2 to V3 boot error (571) esp_core

[spyder0069]

Board

ESP32 Dev Module

Device Description

Custom pcb

Hardware Configuration

Yes

Version

latest master (checkout manually)

IDE Name

Arduino IDE

Operating System

Windows 10

Flash frequency

80Mhz

PSRAM enabled

no

Upload speed

115200

Description

Device that was previously programed with V2 (most likely 2.0.5) after being flashed with v 3.0.2 on the bootup serial I get

ts Jun 8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1448
load:0x40078000,len:14844
ho 0 tail 12 room 4
load:0x40080400,len:4
load:0x40080404,len:3356
entry 0x4008059c
E (571) esp_core⸮3�յ⸮}⸮⸮⸮͡⸮��⸮ɕ⸮"յ⸮⸮"⸮х⸮�⸮⸮⸮⸮⸮2⸮⸮⸮⸮⸮⸮jR4�⸮kV�+�W��⸮,ZY⸮⸮⸮][O⸮⸮⸮&YMX⸮⸮'

Everything seems to work but I do not know what is going on with this esp_core message.
What I have found is if I do a full erase on the chip first and flash with V3 the message does not appear.

Anytime I have my original program that was flashed in with esptool.exe and then try to OTA or use arduino to flash over it I get the error. This makes me think maybe its a partition issue? I am using MINIMAL SPIFFS 1.9MB app with OTA/190k SPIFFS

If I start with a simple program on V2 and then flash V3 its fine.

The original additional files (boot_app0.bin, bootloader_qio_870m.bin, main.ino.partitions.bin,main.spiffs.bin) that were used to flash in the original firmware date from 2020.

I appreciate any info on the error and if anyone knows if the partition setup for (MINIMAL SPIFFS 1.9MB app with OTA/190k SPIFFS) has changed?

Sketch

No sketch

Debug Message

E (571) esp_core⸮3�յ⸮}⸮⸮⸮͡⸮��⸮ɕ⸮"յ⸮⸮"⸮х⸮�⸮⸮⸮⸮⸮2⸮⸮⸮⸮⸮⸮jR4�⸮kV�+�W��⸮,ZY⸮⸮⸮][O⸮⸮⸮&YMX⸮⸮'

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[spyder0069]

I turned core debug level to verbose: and it shows the partition info and some additional stats:

E (750) esp_core⸮f0WW�⸮e⸮+⸮�-'H⸮KW�$VW���V,W,⸮,ZY⸮-HV⸮-[Y⸮⸮
Calculated checksum='233d5ab2'
Image checksum='ffffffff'
[ 247][D][esp32-hal-cpu.c:264] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
[ 272][V][esp32-hal-periman.c:160] perimanSetPinBus(): Pin 3 successfully set to type UART_RX (2) with bus 0x3ffbedd0
[ 283][V][esp32-hal-periman.c:160] perimanSetPinBus(): Pin 1 successfully set to type UART_TX (3) with bus 0x3ffbedd0
=========== Before Setup Start ===========
Chip Info:

Model : ESP32
Package : D0WD-Q5
Revision : 100
Cores : 2
CPU Frequency : 240 MHz
XTAL Frequency : 40 MHz
Embedded Flash : No
Embedded PSRAM : No
2.4GHz WiFi : Yes
Classic BT : Yes
BT Low Energy : Yes
IEEE 802.15.4 : No

INTERNAL Memory Info:

Total Size : 316756 B ( 309.3 KB)
Free Bytes : 286164 B ( 279.5 KB)
Allocated Bytes : 23512 B ( 23.0 KB)
Minimum Free Bytes: 280548 B ( 274.0 KB)
Largest Free Block: 110580 B ( 108.0 KB)

Flash Info:

Chip Size : 16777216 B (16 MB)
Block Size : 65536 B ( 64.0 KB)
Sector Size : 4096 B ( 4.0 KB)
Page Size : 256 B ( 0.2 KB)
Bus Speed : 80 MHz
Bus Mode : QIO

Partitions Info:

            nvs : addr: 0x00009000, size:    20.0 KB, type: DATA, subtype: NVS
        otadata : addr: 0x0000E000, size:     8.0 KB, type: DATA, subtype: OTA
           app0 : addr: 0x00010000, size:  1920.0 KB, type:  APP, subtype: OTA_0
           app1 : addr: 0x001F0000, size:  1920.0 KB, type:  APP, subtype: OTA_1
         spiffs : addr: 0x003D0000, size:   128.0 KB, type: DATA, subtype: SPIFFS
       coredump : addr: 0x003F0000, size:    64.0 KB, type: DATA, subtype: COREDUMP

---

Software Info:

Compile Date/Time : Jul 3 2024 14:24:16
Compile Host OS : windows
ESP-IDF Version : v5.1.4-358-gbd2b9390ef-dirty
Arduino Version : 3.0.2

Board Info:

Arduino Board : ESP32_DEV
Arduino Variant : esp32

[spyder0069]

After much testing I have determined this is not a V3 error. I have narroed it down to something with the partition setup I think. Originally my device used spiffs and I flashed the device using esptool.exe with the following command:

esptool.exe --chip esp32 --port COM7 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0x1000 bootloader_qio_80m.bin 0x8000 Main.ino.partitions.bin 0xe000 boot_app0.bin 0x10000 myapp.bin 0x3d0000 Main.spiffs.bin

Now if I try to flash that with a firmware compiled with 2.0.17 using that same command line I get the core dump line but the chip continues to work.

If I erase the chip and flash the same firmware with esptool.exe removing the spiffs reference then I no longer get the dump.

esptool.exe --chip esp32 --port COM7 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0x1000 bootloader_qio_80m.bin 0x8000 Main.ino.partitions.bin 0xe000 boot_app0.bin 0x10000 myapp.bin

I am not sure if maybe the reserved space for spiffs changed over the different revisions? Once I flash firmware with my spiffs reservation causing the crash I continue to get the crash even if you remove the reference. The chips is stuck until you do an erase.

So my problem is how do I move forward. I no longer need spiffs in my project but my devices were flashed with that address reservation. The problem recreates if I try to OTA a new file to them.

Appreciate any advice! :^)

[spyder0069]

I have narrowed it down. I am still running 2.0.17 just as an example and it doesn't matter what code I have. If I flash the device with the original spiffs.bin it gives the error on startup. This spiffs file had 3 files in it. The file size on the old spiffs.bin shows 192kb or 196,608 bytes. I sort of remember back in 2020 when I created that file that it always created a 192K spiffs file. When I try to create a new spiffs file with the same 3 files I get the error:

[SPIFFS] data : C:\SPYDERROBOTICSDATA\spyderrobotics\FIRMWARE\arduino\SKETCHBOOK\SPYDERWEB_12\JUNK\Main\data
[SPIFFS] start : 3997696
[SPIFFS] size : 128
[SPIFFS] page : 256
[SPIFFS] block : 4096
/jquery.min.js
/spyderweb_logo_left.gif
/spyderweb_logo_right.gif
SPIFFS_write error(-10001): File system is full.

error adding file!

SPIFFS Create Failed!

If you select the 3 files in windows it shows file size to be 92.2KB (94,485 bytes).
If you remove one of the files it will create a new spiffs file.

If I flash with the new spiffs file on an erased chip (doesn't matter what program) everything is fine.
Trying to flash with the new spiffs file over the top of a chip that had the previous larger spiffs file does not correct the core dump error.

So in summary:
The partition scheme I am using is the Minimal Spiffs which shows 190KB. Was this ever 192K previously and that is why it generated the 192K spiffs file that I have?

I am thinking there was some difference with the partition scheme in the past which allowed that file to work and something in later revisions of the esp core that choke when trying to flash over the previous firmware.

Other than the error on boot the device appears to be working but I don't know if there are other effects yet.

Since I have many devices out there with this original configuration it would be nice to find a solution that can fix it via OTA. I did try flashing code that enabled spiffs and did a format. The format was successfull but it still had the core error on boot.

[Xylopyrographer]

Haven't read everything in detail but it does seem like a partition mismatch between what is deployed and what is built using latter versions of core 2.0.x and the newer core 3.0.x.

Solution, if this is the case, is to build the new version using the partition table from what is deployed.

To find that table, a couple of options:

1. go back through the core v2.0.x releases and grab the version used to build the deployed software. In the ../esp32/hardware/esp32/[CORE_VERSION]/tools/partitions folder you'll find all the partition tables. Grab the default from there.
2. if you have an device with the old version, use esptool.py to extract the table from the flash of that device and then convert that to a .csv file.

Once you have the table in .csv format, make sure the file name is partitions.csv and place it in the same location as your sketch .ino file.

Then when the software is compiled, the build system will use that table.

Caveat:
Older versions of the arduino-esp 32 core had a smaller bootloader and the user part of the partition table started at 0x8000. With newer cores, the boot loader is larger and so the user part now starts at 0x9000.

Meaning, you may need to customize the old portion table to compensate, but give it a whirl and see.

Docs on esp tool.py, converting to and from .bin and .csv formats, and partition tables in general is over on the espressif ESP-IDF site.

There are other things that might come in to play but one step at a time.

[spyder0069]

I was able to extract the partitions.csv from my original. After more testing with 3.0.2 installed and a basic program I get the core dump message if I have the old partition.csv in my sketchbook folder. If I remove the partitions.csv from my folder, erase the chip, and compile/flash, with the same selected partition setup I no longer get the core dump message. So somewhere along the line the esp32 core changed breaking compatibility with my partition setup. I looked at the difference in the two files and attached an image

The green partition setup is what 3.0.2 creates and the yellow is what my original project partition setup is. What I notice is the addition of the coredump partition but also the spiffs file size changes. Keep in mind this these were built with MINIMAL SPIFFS 1.9MB app with OTA and 190K spiffs. So what I think is happening is the coredump error on boot is because there is no coredump partition in my partitions.csv. But what also concerns me is the spiffs size is different between my old one. I am not great with conversions but with the new (green) partition setup is it actuatlly only giving 131K of space for spiffs and not the 192K? Seems like it cut down spiff space to make room for the coredump partition. But it still shows 190K in the description. Is there a way to prevent arduino from creating the coredump partition?

[spyder0069]

So in thinking about this the coredump error is because the coredump partition doesn't exist. Which in itself doesn't seem to be a problem because my code still is at the correct positions in flash. The challenge I guess is how can you update a device via OTA with code written in 3.0.2 to a device that does not have a coredump partition? OTA with the main .bin file isn't going to change partition structure correct? This is why it seems I just need the ability to disable the use of coredump to flash.

[spyder0069]

I found this config CONFIG_ESP_COREDUMP_TO_FLASH_OR_UART that has the option of

None (ESP_COREDUMP_ENABLE_TO_NONE)

I think this would eliminate my error but I am not sure if this is possible in the arduino core?

These was also CONFIG_ESP_COREDUMP_CHECK_BOOT that shows the default as:

Yes (enabled) if ESP_COREDUMP_ENABLE_TO_FLASH

Maybe this check system also needs to be disabled?

[Xylopyrographer]

I don't believe the core dump partition is mandatory.

Does the code run if you use the table without that partition?

Didn't go back through everything, but what message are you seeing? If it's only a warning then no problem.

[spyder0069]

Yes. The code works. I just get a error out the serial on boot "E (571) esp_core⸮3�յ⸮}⸮⸮⸮͡⸮��⸮ɕ⸮"յ⸮⸮"⸮х⸮�⸮⸮⸮⸮⸮2⸮⸮⸮⸮⸮⸮jR4�⸮kV�+�W��⸮,ZY⸮⸮⸮][O⸮⸮⸮&YMX⸮⸮'" Functionally I could just ignore it as the error doesn't stop the code from running. It would just be nice to clean it up and also get and explanation why it seems the spiffs allocation does not match. If I am correct on that then that should be addressed because you can't actually use 190K as shown in the description if there is only 131,072 available per the partitions file that 3.0.2 is creating.