error v1.18 build esp32 GENERIC_SPIRAM with user modules

[erongd]

I am trying to build ESP32 GENERIC_SPIRAM board with st7789_mpy native driver.

I build using the v1.18 branch, did it with esp-idf 4.2, 4.2.2 and 4.3.2.

Here is the command I use to build.
make USER_C_MODULES=~/st7789_mpy/st7789/micropython.cmake BOARD=GENERIC_SPIRAM -j 2

Here is the error I am getting with 4.2 and 4.2.2.
Project build complete. To flash, run this command:
/home/erong/.espressif/python_env/idf4.2_py3.7_env/bin/python ../../../esp-idf-4.2/components/esptool_py/esptool/esptool.py -p (PORT) -b 460800 --before default_reset --after hard_reset --chip esp32 write_flash --flash_mode dio --flash_size detect --flash_freq 40m 0x1000 build-GENERIC_SPIRAM/bootloader/bootloader.bin 0x8000 build-GENERIC_SPIRAM/partition_table/partition-table.bin 0x10000 build-GENERIC_SPIRAM/micropython.bin
or run 'idf.py -p (PORT) flash'
bootloader @0x001000 21600 ( 7072 remaining)
partitions @0x008000 3072 ( 1024 remaining)
application @0x010000 1584560 ( -11696 remaining)
ERROR: application overflows allocated space of 1572864 bytes by 11696 bytes
make: *** [Makefile:35: all] Error 1

Here is the error I am getting with 4.3.2.
/home/erong/.espressif/tools/xtensa-esp32-elf/esp-2021r2-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: micropython.elf section .iram0.text' will not fit in region iram0_0_seg'
/home/erong/.espressif/tools/xtensa-esp32-elf/esp-2021r2-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: IRAM0 segment data does not fit.
/home/erong/.espressif/tools/xtensa-esp32-elf/esp-2021r2-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: region `iram0_0_seg' overflowed by 392 bytes
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
ninja failed with exit code 1
make: *** [Makefile:34: all] Error 2

The same works fine with build for ESP32 GENERIC and ESP32S2 board. And it was working fine with v1.17 branch. Would appreciate any help.

[russhughes]

The application partition is too small to hold the firmware. However, you can change the ESP32 partition sizes by editing the boards partitions.csv file. Some boards use other partitions files, but the default file for GENERIC boards is in ports/esp32.

The partitions.csv file for a 4MB flash device looks like this:

# Notes: the offset of the partition table itself is set in
# $IDF_PATH/components/partition_table/Kconfig.projbuild.
# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x6000,
phy_init, data, phy,     0xf000,  0x1000,
factory,  app,  factory, 0x10000, 0x180000,
vfs,      data, fat,     0x200000, 0x200000,

The factory partition is where the MicroPython firmware resides. This partition must be large enough for the MicroPython firmware. The application partition must also start at an offset aligned to 0x10000 (64K). Finally, the vfs partition is where the virtual file system is located.

The factory partition starts at 0x10000 (aligned to 64K as required) and has a size of 0x180000. This places the end of the factory partition at (0x10000 + 0x180000) or 0x190000. The next partition is the vfs partition, it starts at 0x200000 (2MB) and has a size of 0x200000 (2MB). This leaves 0x200000 - 0x190000 = 0x70000 or 448K bytes of unused flash between the end of the factory partition and the start of the vfs partition. Adding the unused flash to the factory partition results in a new factory partition size of 0x180000 + 0x70000 = 0x1F0000 or 64K less than 2MB.

Modified partitions.csv file contents

# Notes: the offset of the partition table itself is set in
# $IDF_PATH/components/partition_table/Kconfig.projbuild.
# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x6000,
phy_init, data, phy,     0xf000,  0x1000,
factory,  app,  factory, 0x10000, 0x1F0000,
vfs,      data, fat,     0x200000, 0x200000,

If more 'factory' flash space is required, you can decrease the vfs partition size and increase its offset by the same amount, then add the newly freed space to the factory partition.

Modified partitions.csv file contents for a 3MB factory partition and a 960K virtual file system
partition.

# Notes: the offset of the partition table itself is set in
# $IDF_PATH/components/partition_table/Kconfig.projbuild.
# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x6000,
phy_init, data, phy,     0xf000,  0x1000,
factory,  app,  factory, 0x10000, 0x300000,
vfs,      data, fat,     0x310000, 0xF0000,

Once you have your partition sizes worked out, build the MicroPython firmware. When you modify the partition table, you must erase the flash before you flash the new firmware to your device.

make PORT=/dev/ttyUSB0 erase
make PORT=/dev/ttyUSB0 deploy

[erongd]

Thanks this works.

[volgunov-ad]

I have the same error with 8MB flash and two OTA partitions

nvs data nvs 0x9000 0x4000
otadata data ota 0xd000 0x2000
phy_init data phy 0xf000 0x1000
ota_0 app ota_0 0x10000 0x300000
ota_1 app ota_1 0x310000 0x300000
vfs data fat 0x620000 0x1E0000

[volgunov-ad]

When I make GENERIC everything is fine, but if I try to do GENERIC_SPIRAM the following error comes out

[erongd]

When I make GENERIC everything is fine, but if I try to do GENERIC_SPIRAM the following error comes out
You will have to increase the size of ota_0 and ota_1 partition like @russhughes mentioned above. As these partitions is where micropython firmware stored.

Looks like there are some workarounds for hardware bug and performance improvement done in 1.18 for ESP32 SPIRAM handling that have increased the size of the micropython footprint specifically for ESP32 with SPIRAM.

[volgunov-ad]

I found the solution here micropython/micropython#8202

[volgunov-ad]

add the following to sdkconfig.board:
CONFIG_COMPILER_OPTIMIZATION_SIZE=y
CONFIG_COMPILER_OPTIMIZATION_PERF=n