Memory leak after opening maildir mailbox #4087
Comments
|
I not sure that NeoMutt is leaking, but it does seem to be using quite a lot of memory. When you open NeoMutt, obviously it allocates memory for all the mailboxes in the sidebar and all the emails in the open mailbox. I did quick test whilst watching the memory in I created two macros: macro index <f5> "<change-folder>-<enter>"
macro index <f6> "<f5><f5><f5><f5><f5><f5><f5><f5><f5><f5>"Then, I opened a small folder, then opened a very large folder. I hit <f6> five times and waited. The memory usage fluctuated as folders were opened and closed, but crucially it didn't creep upwards. I'll leave this open and keep thinking |
|
Maybe some more data to compare: Memory use after opening initial small mailbox only without opening big one is negligible and less than 2MB. Memory use after opening small mx -> big mx -> small mx Memory use after opening small mx -> big mx -> big mx -> small mx: So it's seems pretty linear with 30MB allocated more per big mx open and matches pretty well allocation of 157MB after opening big mx 5 times that was shown in initial comment. |
|
OK, thanks. If you've got the time, please could you build with the Address Sanitizer If there are any leaks, it prints out a report on exit telling us exactly where the allocations occurred. |
|
I don't think asan would show anything different than The reason why I used valgrind's Anyway I found the issue: Lines 1245 to 1249 in fa83255 This logic is wrong as far as I can tell. Calculated rounded req_size is assigned to m->email_max which will always be selected in above MAX() and further increased by 25 due to increment and it will only keep on growing by 25 with each call. Since it is called per each email and every call allocates 25 more of sizeof(void*) + sizeof(int) which on my machine happens to be 12b, with 108260 emails that I have in big mailbox gives 108260*25*12=32478000. The 30MB leak that I'm seeing.
|
|
I guess the growing should occur only if original |
|
Damn! You're good! Thanks for the debugging. |
|
Made an attempt to fix the issue in #4089. Memory use in situation from description (smal mx -> 5x big mx -> small mx)
Although use when having big mx open is possibly still concerning: 500MB. To be analyzed independently though. I will create one more PR for reducing allocation overhead a little. |
|
Thanks for diving in; I'll have a play in a second.
Long ago, I had an idea to create a |
Fixes neomutt#4087 - Fix overallocation of memory - Reduce number of re-allocations Reduce mx allocation overhead by gradually increasing grow step so far memory allocated for emails grew by capacity of 25 entries per each call. for big mailboxes this causes quite a lot of reallocations. change the logic to initially allocate 25 entries while further expansions double current capacity. cap number of new entries to 1000 though.
Fixes #4087 - Fix overallocation of memory - Reduce number of re-allocations Reduce mx allocation overhead by gradually increasing grow step so far memory allocated for emails grew by capacity of 25 entries per each call. for big mailboxes this causes quite a lot of reallocations. change the logic to initially allocate 25 entries while further expansions double current capacity. cap number of new entries to 1000 though.
Expected Behaviour
Memory use remains constant when reopening over and over same, unmodified maildir mailbox.
Actual Behaviour
Memory use grows with every open.
Steps to Reproduce
Observe difference in memory between first and last step.
It appears to be caused by memory not being freed:
neomutt/maildir/shared.c
Line 95 in fa83255
as indicated by
massifafter step 4:I'm also not sure if having
470MBallocated when viewing ~100K mailbox is fine or not but if needed I can sharemassifbreakdown of it too.How often does this happen?
When did it start to happen?
I first observed with version
20231006but didn't confirm if it's really the first version it occurred in.NeoMutt Version
The text was updated successfully, but these errors were encountered: