[TW#18774] memory leak during float operation?

[Mr-Techtron]

static void test(void *pv){
float d = 5.1423;
do{
d *= 2;
printf("Heap memory:%d\n", xPortGetFreeHeapSize());
vTaskDelay(1000);
}while(1);
}

When I create this test task, where there is the float operation going on, simply multiplying by 2, then there is the gradual decrease in heap memory. And only even if I give this task the stack depth of 1000, there is the stack overflow. Isn't stack depth of 1000 is more than enough for this task? Have I missed something?

[chanibal]

Check if this is not printf that does this. Try sprintf to a fixed buffer and then puts it.

[Mr-Techtron]

Hi chanibal, even if I followed your instruction, there is still memory leak. The heap size gets gradually decreased with increasing time

[mahavirj]

@Mr-Techtron Are there any other tasks (or services) running in your application that could potentially be allocating dynamic memory? Do you get similar result if you test above code in some standalone application like hello_world demo from IDF?

[Mr-Techtron]

Hi Mahavirj, no! this is the only task I'm running. I've already done some test with simply printing "hello world" and also performed the above multiplication operation in 'int' datatypes, there is no memory leak. Memory leak only happened when I performed the float operation.

[projectgus]

Hi @Mr-Techtron,

printf() uses quite a bit of stack space, which is why you need more than a 1000 byte task stack if using printf().

I put your code into a test app as shown here, with default "sdkconfig" settings:
https://gist.github.com/projectgus/d7d3c74193f2f402248e2709d2f80b18

Output is as follows. Similar on both IDF v3.0-rc1 and current master branch:

I (240) cpu_start: Pro cpu start user code
I (258) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
Heap memory:291780
Heap memory:292740
Heap memory:298328
Heap memory:298328
Heap memory:298328
(etc)

The very first printf() in a task will allocate some memory, due to newlib stdout/stderr buffering. After this, no additional memory is allocated.

If you have a test app which behaves differently, can you please supply the full source, configuration settings (sdkconfig file), and let us know which version of ESP-IDF you are using?

[Mr-Techtron]

minicom1.txt

sdkconfig.txt

main.txt

Last commit of esp32-idf : c3bec5b
I've attached the necessary files of my simple test program. I still encounter the memory leak. Have I done something wrong?

[projectgus]

Thanks for the extra details. You haven't done anything wrong.

The root cause here is that some memory is being allocated every time printf() prints a longer and longer floating point number. This is because a buffer is needed in the printf() implementation (part of the newlib libc) to manage this extra length. You'll find the free heap usage only ever goes down after the number being printed has grown some extra digits (then it stays stable for a while, because the buffer is big enough to handle more than one additional digit, but eventually the ever-growing number gets too long for that buffer and it has to grow again...)

Most of this storage is not "leaked" as such, it's associated with the newlib "reent" structure for the calling task. So the memory will be freed if/when the task is deleted. It does look like newlib may over-allocate these buffers a little and then free them all when the task is deleted, so there may be some small savings we could get by rewriting some of this - although also possibly not the case, I haven't looked closely enough at the implementation of the floating point printf() code (the call stack is _printf_float -> __cvt -> __dtoa_r which triggers extra allocations).

It does look like there may be some memory used in the "reent" structure, associated with printing floats, which isn't correctly freed when the task is deleted (something like 100 bytes per task, but only if the task printed a lot of floats). Will look into this further.

(I also found a bug in the heap tracing while investigating this issue, allocations made by newlib code in ROM weren't based traced. Will push a fix for this.)

[Mr-Techtron]

Thank you Angus for the brief explanation :)

[projectgus]

Closing as the heap tracing in newlib issue is fixed, and the other part is a WONTFIX in newlib.