Memory leak on skvideo.measure.scenedet

[erichi]

Hi, i'm trying to detect scenes using luminance algorithm on 8 minutes video, and after using all available memory process gets killed. I only get results on short ~1min videos.
I'm running script on my local machine.
Is there any solution for this kind of problem or i need to deploy it on cloud to work with long videos?
Thank you!

[beyondmetis]

Based on this issue, I realized there was an inefficiency in the histogram-based scene detection (the default setting). The code converted the entire video to luminance in one pass, creating a copy in memory. This is now avoided. I just pushed and tested the fix, so you should not have that problem at least.

At some point, memory would become a concern anyway, since the scene detector requires the whole chunk to be provided at once. I realize this is incredibly inefficient, since the algorithm only makes comparisons between 2 frames. For larger videos, where you don't want to load the entire video at once, you could provide pairs of frames to the scene detection function and keep track of the frames that are returned ([0] is always returned as a scene boundary, but you should probably discard that).

[erichi]

Thanks so much @beyondmetis ! I've tested the fix on the same video and now it works.
And thanks for the recommendation for larger videos.