Memory leak while calling "get_key_frames_from_video" function in a for loop

[RahulnKumar]

While calling the function "get_key_frames_from_video (path_to_video, no_of_frames)" in a for loop, RAM usage goes on increasing.
For just 10 small videos ( < 1min ) in loop, RAM usage went beyond 6GB

[mayank10j]

What is the average video size on which you are doing this processing ?
Did you tried calling extract_keyframes_from_videos_dir function ?
Also what is your system memory ?
Also it would be helpful if you can share your Machine Environment: OS , OS version, CPU etc.

[RahulnKumar]

These are my system details :
OS : Ubuntu 18.04
RAM : 12 Gb
Python 3.6.9
Same issue was there while I was trying to run my script in colab.
Average video size was 5 MB.
Yes, I tried calling extract_keyframes_from_videos_dir function also, same issue was there too.
For a total of 20 videos (108 MB) in folder, % RAM usage stats are as follows:
using function extract_keyframes_from_videos
ram_usage_ = [34.6, 36.2, 36.6, 37.3, 38.1, 39.1, 39.4, 40.5, 41.4, 42.5, 43.4, 44.4, 45.2, 47.0, 47.4, 47.9, 48.6, 49.7, 50.6, 51.6]
using function extract_keyframes_from_videos_dir
ram_usage = [34.1, 36.0, 36.5, 37.4, 38.1, 39.1, 39.5, 40.7, 41.4, 42.3, 43.3, 44.2, 45.5, 46.6, 46.9, 47.7, 49.2, 49.7, 50.6, 51.5]

Above list is increase in RAM % usage while these functions are called in a for loop.

[mayank10j]

For one frame of A typical 720p video with resolution of 1280 x 720 loaded in memory takes around 3*1280 x 720 = 2764800 = 2.7648 MB in RAM
the maximum number of frames kept in memory (configurable by Katna.config.FrameExtractor.max_frames_in_chunk = 2500 ) 2500 frames for a typical 30fps video = 84 Sec of video content.

So for 2500 frames RAM usage would be 2500*2.7648 MB = 6912 MB ,
Video processing is a RAM intensive task. There is no memory leak in code.
If you still want to reduce RAM usage change this config variable Katna.config.FrameExtractor.max_frames_in_chunk to a lesser value.

[RahulnKumar]

Still I'm not able to limit RAM usage. It goes on increasing when I'm calling this function in a for loop. And the RAM is free only when I restart the jupyter kernel or when python script is finished running.
This time I used following config for limiting RAM as you said :
config.FrameExtractor.max_frames_in_chunk = 50
But then also % RAM usage went on increasing as follows
ram_usage = [31.7, 33.0, 33.7, 35.1, 36.2, 36.6, 37.8, 38.8, 39.4, 41.4, 42.7, 43.3, 43.9, 45.4, 46.3, 48.3, 49.9, 51.9, 52.1, 53.0, 54.0, 55.6, 56.6, 57.9, 59.0, 59.7, 60.3, 60.8, 61.1, 62.4, 63.3, 64.6, 66.2, 67.4, 68.9, 69.7, 70.4, 71.5, 72.9]
This time my RAM went from 3GB to 8GB ( around 40 videos) while running my script but max RAM usage should be 50*2.78 MB = 137 MB.
I think frames of a particular video is not released from memory while we iterate through the videos and thats why RAM usage goes on increasing.

This is the script I'm running : Python Script , Jupyter Notebook
Please see if I'm doing something wrong.

[mayank10j]

2.68 MB is RAM Usage if you load a single Frame of a video in RAM, typically
for a decent sized (720p) video 2500 frames are loaded in memory at once, so RAM usage should be 2500* 2.68 MB approximately which is around 6.7 GB.
50 Videos it should be 6.7 GB * 50 = 335 GB, But fortunately code does not do processing on all 50 GB once, it does processing on videos one by one so maximum is again around 6 GB only.

[RahulnKumar]

Sorry to disturb you again.
This time I tried running it on VM with 64 GB RAM. And only this key_frame_extraction script was running on the machine. But this time also RAM size went on increasing with every next vedio in the for loop.

This time also % RAM usage was increasing :
[5.5, 7.6, 9.5, 11.5, 13.6, 17.0, 19.1, 21.2, 24.7, 26.8, 29.0, 32.5, 34.6, 36.9, 40.8, 43.4]
I don't know where but there is some issue somewhere because according to you max ram usage should be around 6 GB but here it's always increasing and went upto 30 GB of RAM (after which I stopped script).

[thealokkr]

Hi @RahulnKumar , thanks for reaching out. I was wondering if you would like to contribute to this project with this issue fix. We are bit overwhelmed with our office work at the moment and might not be able to investigate immediately.

[RahulnKumar]

Ok, sure. I'll let you know when fixed.

[mayank10j]

Can you also tell me which operating system and OS version you are using ?
Also which python version you are using ?
I am unable to replicate the issue in MacOSX

[mayank10j]

Also if possible can you put
import gc

and then
call gc.collect()
in for loop and see if memory usage explosion is still happening ?

[RahulnKumar]

Yes, I tried garbage collector earlier also. Again I tried, still facing the same issue .
These are some specifications you asked:
OS : Ubuntu 18.04
Python version : 3.6.9

[mayank10j]

This memory leak issue is fixed in katna version 0.9.0, kindly test and confirm.

[RahulnKumar]

Yes Working perfectly now. 😊
But there is still a very small issue.

def extract_video_keyframes(self, no_of_frames, file_path, writer):
        """Returns a list of best key images/frames from a single video.

        :param no_of_frames: Number of key frames to be extracted
        :type no_of_frames: int, required
        :param file_path: video file location
        :type file_path: str, required
        :param writer: Writer object to process keyframe data
        :type writer: Writer, required
        :return: List of numpy.2darray Image objects
        :rtype: list
        """
        top_frames = self._extract_keyframes_from_video(no_of_frames, file_path)
        writer.write(file_path, top_frames)
        print("Completed processing for : ", file_path)  

You forgot to add return top_frames in this function and that is causing problem 🤔. So None is returned instead of list of images while calling this function.

[thealokkr]

It is breaking change from the last version. The function not only extracts but saves it on the disk as well via the writer. Documentation has been updated with the details
https://katna.readthedocs.io/en/latest/tutorials_video.html

[RahulnKumar]

I got it. Thanks. 🤗