Combining thousands of small clips into one file

[ppisecky]

I have thousands (7709) of 3 second .mov clips that I need to combine into a single video. I am currently looking for the easiest and fastest way to do this. I think that doing that in an editor would be a pain in the *** so I am looking for other options.

All of the clips have the same resolution, duration and frame rate. My initial assumption was, that trying to load all of them into VideoFileClip objects and using concatenate_videoclips wouldn't simply work very well due to memory issues and so on. So I am trying to do it one by one.

I chose the simplest algorithm I could think of:

1. Check if a video made combining several clips already exists. combined_file
2. If it does append another clip to it.
3. Otherwise merge the first two clips.
4. Save the new combined_file

Expected Behavior

A video combining all of the clips from 1 to N.

Actual Behavior

A video does get made, but every time I overwrite it the first section of the video (i.e. everything apart of the newly added clip) gets frozen. What I mean is that it looks like there is only one frame for the original duration of the combined clip and only the newly appended clip plays. Audio is fine.

Steps to Reproduce the Problem

from moviepy.editor import VideoFileClip, concatenate_videoclips
import os.path

# All of the clips are named [n].mov
start = 1
end = 3

clip_path = ".\\clips\\"
combined_filename = "render_test.mov"

# Get a clip filename: path to clips folder + i + .mov extension
def getClipFilename(i):
    return clip_path + str(i) + ".mov"

# Keep appending clips to the render_test.mov video. If it doesn't exist combine the first two clips.
for i in range(start, end+1):
    print("Appending", i)
     
    if os.path.isfile(combined_filename):
        combined_file = VideoFileClip(combined_filename)
        clip = VideoFileClip(getClipFilename(i))
    else:
        # I realize that the way this is done means that the second clip will be added twice. 
        # That's not my concern atm.
        print("Main file not found. Merging first two clips.")
        combined_file = VideoFileClip(getClipFilename(i))
        clip = VideoFileClip(getClipFilename(i+1))
        
    final_clip = concatenate_videoclips([combined_file, clip])
    final_clip.write_videofile(combined_filename, codec='libx264')

Specifications

• Python Version: 3.6.4
• Moviepy Version: Latest via pip install
• Platform Name: Windows 10

As a side question - is there a way to limit the resources used by the script? Using this moviepy makes my CPU usage go to 100% and I would very much like to avoid that due to cooling concerns.

Thanks in advance for any help.

[ppisecky]

I thought more about this problem and I think that this is a pretty bad approach anyway due to the fact that I would have to be writing the same information to the disk over and over again and it would be increasing all the time. Maybe I could make several smaller combined clips made of batches of 100-200 files and combine those in the end? The combined size of all the clip files is about 26GB. So I'm not sure how to deal with such a large amount of data.

[tburrows13]

To be honest, I think using a graphical editor might actually work. Have you tried? I’d have thought that most editors can import a whole folder at a time. It might take a while, but that’s a given. If you do use moviepy, make sure that you have a 64-bit installation of python. 32-bit installs are limited to 4GB of RAM.

[keikoro]

@ppisecky Did you manage to find a workaround for the issue? Can we close it?

[ppisecky]

@keikoro Oh yeah sorry. I used a graphical editor in the end