[tools/idf_size.py][components/esp_system/ld/esp32s3/memory.ld.in] Wrong ouput of idf.py size (IDFGH-8504)

[sbithinx]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v4.4.2

Operating System used.

Linux

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

No response

What is the expected behavior?

Correct statistics about the memory usage of ESP32-S3-based project, when idf.py is called with the option size, size-files or size-components.

What is the actual behavior?

Currently the output of idf.py size-files for our ESP32-S3-based project with 16 KB as 'CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE` looks like this:

Total sizes:
Used static IRAM:   94242 bytes ( 331742 remain, 22.1% used)
      .text size:   93215 bytes
   .vectors size:    1027 bytes
Used stat D/IRAM:  189705 bytes ( -16777 remain, 109.7% used) Overflow detected!
      .data size:   23577 bytes
      .bss  size:  166128 bytes
Used Flash size : 1184283 bytes
      .text     :  913231 bytes
      .rodata   :  270780 bytes
Total image size: 1302102 bytes (.bin may be padded larger)

It can be assured, that our software is not using ~16 KB more than the available RAM. I spend a lot of hours tracing down the issue in the tools/idf_size.py script.
It seems, that idf_size.py is not able to handle, if only 16 KB are configured, instead of the maximal available 32 KB, as CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE.

If 32 KB are configured as CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE. The output looks better, but another issue can be seen:

Total sizes:
Used stat D/IRAM:  283947 bytes (  61909 remain, 82.1% used)
      .data size:   23577 bytes
      .bss  size:  166128 bytes
      .text size:   93215 bytes
   .vectors size:    1027 bytes
Used Flash size : 1184259 bytes
      .text     :  913231 bytes
      .rodata   :  270756 bytes
Total image size: 1302078 bytes (.bin may be padded larger)

Keep in mind, this the output for our project with just CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE changed. If the reported bytes are added up:

Used stat D/IRAM: 283947 + 61909 remain = 345856 Bytes

The ESP32-S3 has 512 KB integrated RAM and 345856 Bytes != 512 KB, even if the configured CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE is subtracted. This can be traced down to section Memory Configuration in the MAP-file:

Memory Configuration

Name             Origin             Length             Attributes
iram0_0_seg      0x0000000040378000 0x0000000000054700 xr
iram0_2_seg      0x0000000042000020 0x00000000007fffe0 xr
dram0_0_seg      0x000000003fc88000 0x0000000000054700 rw
drom0_0_seg      0x000000003c000020 0x00000000007fffe0 r
rtc_iram_seg     0x00000000600fe000 0x0000000000002000 xrw
rtc_data_seg     0x00000000600fe000 0x0000000000002000 rw
rtc_slow_seg     0x0000000050000000 0x0000000000002000 rw
*default*        0x0000000000000000 0xffffffffffffffff

As you can see, the reported size of iram0_0_seg and dram0_0_seg is 0x54700 which is the same value as calculated earlier: 345856 Bytes. I debugged the idf_size.py script to find the root for the reported information.
It comes from the components/esp_system/ld/esp32s3/memory.ld.in linker file:

#define SRAM_IRAM_START     0x40370000
#define SRAM_DIRAM_I_START  0x40378000
#define SRAM_IRAM_END       0x403CC700 /* Please refer to ESP32-S3 bootloader.ld for more information on this */
#define I_D_SRAM_OFFSET     (SRAM_DIRAM_I_START - SRAM_DRAM_START)

#define SRAM_DRAM_START     0x3FC88000
#define SRAM_DRAM_END       (SRAM_IRAM_END - I_D_SRAM_OFFSET)  /* 2nd stage bootloader iram_loader_seg start address */
#define I_D_SRAM_SIZE       (SRAM_DRAM_END - SRAM_DRAM_START)


#define ICACHE_SIZE         0x8000
#define SRAM_IRAM_ORG       (SRAM_IRAM_START + CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE)
#define SRAM_IRAM_SIZE      (I_D_SRAM_SIZE + ICACHE_SIZE - CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE)

...

  /* IRAM for PRO CPU. */
  iram0_0_seg (RX) :                 org = SRAM_IRAM_ORG, len = SRAM_IRAM_SIZE

https://github.com/espressif/esp-idf/blob/5ee663d5926bcb7e09c64f1eb8500f94e1bdc18c/components/esp_system/ld/esp32s3/memory.ld.in#L37:L49

https://github.com/espressif/esp-idf/blob/5ee663d5926bcb7e09c64f1eb8500f94e1bdc18c/components/esp_system/ld/esp32s3/memory.ld.in#L69:L70

If the calculation is extended:
SRAM_IRAM_SIZE = ((SRAM_IRAM_END - (SRAM_DIRAM_I_START - SRAM_DRAM_START)) - SRAM_DRAM_START) + ICACHE_SIZE + CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE

and the absolute values are put in place:
SRAM_IRAM_SIZE = ((0x403CC700 - (0x40378000 - 0x3FC88000)) - 0x3FC88000) + 0x8000 - 0x8000 = 0x54700

At this point I really don't understand why two different address ranges are used mixed here. Escpecially because this step:

I_D_SRAM_OFFSET = SRAM_DIRAM_I_START - SRAM_DRAM_START = 0x40378000 - 0x3FC88000 = 0x6F0000

Why is the DRAM-Start-Address in the data bus address range subtracted from the D/IRAM-Start-Address in the instruction bus address range? This does not really make sense to me.
For CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE set to 32 KB the calculated SRAM_IRAM_SIZE should be at least 480 KB (Internal SRAM 1 + Internal SRAM 2), 416 (Internal SRAM 1) or 512 KB (Internal SRAM 0 + Internal SRAM 1 + Internal SRAM 2), but definitely not 345856 Bytes.

Steps to reproduce.

1. Call idf.py with option size-files with 16 KB CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE for target ESP32-S3
2. Call idf.py with option size-files with 32 KB CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE for target ESP32-S3
   ...

Build or installation Logs.

No response

More Information.

No response

[igrr]

Thank you for the report! We have noticed this issue internally as well. We'll let you know once the fix is available.

[DNedic]

@sbithinx I have looked into this issue, the value of 345856B is infact the correct value for DIRAM, 512K is the sum of all RAM regions, while DIRAM is only 416KB:

esp-idf/tools/idf_size_yaml/esp32s3_data_info.yaml

Line 10 in 439a709

length: 0x8000 + 6 * 0x10000

however even that is not entirely available as the second stage bootloader is using parts of it:

esp-idf/components/esp_system/ld/esp32s3/memory.ld.in

Line 39 in 439a709

#define SRAM_IRAM_END 0x403CC700 /* Please refer to ESP32-S3 bootloader.ld for more information on this */

The calculation when cache is 16k is wrong and that is being fixed at the moment.

[sbithinx]

@DNedic Thanks for the update, I wasn't considering the reserved memory by the second stage bootloader. However, it would be nice, if the idf_size.py summary would also report the memory used by the second stage bootloder. Do you agree?

[DNedic]

@sbithinx Thank you for the suggestion!