Possible memory leak #23
Comments
|
Certainly possible. I don't use this anymore and it was maybe the first Go code I wrote, so it is likely very bad at memory. I'm definitely not going to work on fixing it but I would review a PR. |
|
@pawel0987 I am using this library as well and my app got to 40GB memory usage after running for a few days. The issue is from the fft/bluestein.go: there are two map that caches all the bluesteinFactors and bluesteinInvFactors, which gets larger over time. In my use case, my data set is large as well with different input_len most of the time. Matt's implementation was to speed up the calculation by memorizing the bluesteinFactors and bluesteinInvFactors, but it's not memory friendly. I did some modification for my use case and the calculation slows down, but I don't need to worry about high memory usage anymore. |
|
@J1ng06 Would you be able to share the modifications that you made to avoid the memory leak ? Much appreciated |
ghost
mentioned this issue
* Init go module * Fix bluestein cache memory leak through lfu strategy cache with 64 items: According to mjibson#23 there is a memory leak bug in the caching of bluestein factors. This addresses this through an lfu cache with a capacity of 64 items. Based on the input length and thus the size of the computed factors this is sufficient. The max size of the cache can be calculated through: size_factor = (128*2) / 8 = 32 max_size = size_factor * input_len * 64 E.g.: A 1 sec wav file with sample rate of 44.1 kHz and an arbitrary sample size results in 32 * (1 * 44100) * 64 = 86.13 MByte
Invoking
fft.FFT()many times for large wav files (~10MB) couses program to panic because ofout of memory.I have tested my program with and without that single line. Without computing fft it occupies ~300MB for very long time. When I enable fft computing it will be 3GB of RAM occupied pretty fast.
The text was updated successfully, but these errors were encountered: