[Heap Trace Standalone] Support storing records in SPI RAM (IDFGH-8683)

[chipweinberger]

❌ EDIT: WONT MERGE, due to new PR with simpler approch: #10141 ❌

Add support for storing records in SPI RAM. Internal RAM is just used as a temporary buffer.

With this change, amazingly a small internal ram record buffer size of 13 was enough to handle all records! And this is for a very complex app with bluetooth, wifi, console. etc.

With this change, I am able to store all allocations since boot. During development one can leave this enabled, & If they unexpectedly hit a Low Ram condition, they can immediately print all allocations and analyze the problem without needing to reproduce with host mode enabled.

This makes standalone mode 10x more useful, IMO.

Also adds heap_trace_dump_internal_ram_only(), which is often all we care about.

And adds heap_trace_summary() so that users can access more useful heap tracing information.

Code:

In this PR, Internal RAM is just used as a temporary buffer. We stream records to SPI ram whenever we get a chance (i.e. at the next heap_trace_X() invocation from a non-IRAM function).

[chipweinberger]

Gonna merge the 2 standalone implementations into one. Give me a a little bit.

[chipweinberger]

Okay, changes are done. I re-tested heap_trace_init_standalone and heap_trace_init_standalone_with_spi_ram and they work as expected.

This should be good to merge.

[Alvin1Zhang]

Thanks for your contribution.

[igrr]

Question, why can't we simply allocate the trace buffer in external RAM? I.e. place heap_trace_record_t into external RAM and call heap_trace_init_standalone?

If the user application doesn't call heap-related functions when cache is disabled, then there shouldn't be a problem accessing external RAM from the heap tracing functions.

[chipweinberger]

@igrr

From esp_heap_trace.h

* @param record_buffer Provide a buffer to use for heap trace data.
*  Must remain valid any time heap tracing is enabled, meaning
 * it must be allocated from internal memory not in PSRAM.

I'd think avoiding the "cache disabled" issue is important.

[igrr]

In ESP-IDF internal components we don't perform heap allocations from interrupt handlers, so there should be no cases of accessing heap functions while cache is disabled. This might only be the case in user code, if a heap related function is called from an ISR. However I don't think there is a need to support these two things simultaneously:

1. heap functions called from ISRs (hence, possibly while cache is disabled)
2. heap tracing into a buffer in external RAM

since 1) is a bad practice and is not recommended anyway.

Adding an assert somewhere in heap tracing code that !(trace buffer pointer is in external RAM && cache is disabled) seems to be sufficient to catch this potential issue. Plus, updating the lines of documentation you have pointed to say that the heap trace buffer can be in external RAM if no heap allocations from IRAM ISRs are performed.

[chipweinberger]

Ok. Can you give more details on how to do this? !(trace buffer pointer is in external RAM && cache is disabled)

[chipweinberger]

I'll close this PR and open a new one without the dual Internal / SPI ram buffers.

[igrr]

Sorry, I should have mentioned that. Probably something along the lines of: spi_flash_cache_enabled() || esp_ptr_internal(ptr)