Precise Splitting

[cjbarth]

Is there any interest in a PR for using ffmpeg to do precise splitting of files? I know the current splitting only works on key-frames.

[Breakthrough]

Hello @cjbarth;

I've been giving this some thought for a while, the only issue I see is the passing of the video encoding parameters properly to ffmpeg (or maybe providing some default values instead). It would be possible to add additional command-line parameters to PySceneDetect, which could be passed to ffmpeg, so this is definitely possible. For each split I assume there will need to be a call to ffmpeg for each individual scene, as opposed to mkvmerge (at least given my use of it) - although this presents opportunities for parallelism.

Indeed though, if you have any suggestions or changes with respect to allowing ffmpeg in addition to mkvmerge, I would be most open to any PR attempting this. While my primary focus is creating a stable API for the next release, I can see this being a very useful feature in the future, and a lot of the legwork has been done with integrating mkvmerge already.

It should also be noted that any implementations of this feature should allow for interchangeability with the libav project, although the command-line tools share an almost identical syntax to those of ffmpeg.

[cjbarth]

I was thinking that some sensible defaults for ffmpeg would be all that is needed. Perhaps a CLI argument to take an integer for qscale, one to take a string for vcodec and one to take a string for acodec (though I'm tempted to leave this defaulted to copy because we are only working with video here).

As for compatibility with the libav project, if we stick to the above mentioned CLI args, then compatibility will be no problem. We can just have python check for ffmpeg first, if it exists, use that, and if it doesn't, try to use avconv.

About threading, ffmpeg tries to thread as much as possible. Therefore, it would be wise to keep the usage of ffmpeg serial from the perspective of python and only start one instance of ffmpeg at a time.

How does that sound?

[Breakthrough]

Sounds excellent to me from an implementation perspective, that sounds like a sensible approach for a first implementation. Can the bitrate and number of passes be modified through the vcodec argument, or is that a separate parameter (or related to qscale)? (I apologize, am not too familiar with ffmpeg). As for acodec, that sounds like a good default value to me as well, and allowing an override would make sense too.

I agree with your comments about threading, scenes should just start to be encoded/outputted sequentially, in the order they are detected. Likewise, I agree with your comments about the threading model, and it sounds good to me. For now it would be best to just start batch processing the results one by one, exactly they are now with mkvmerge (once scene detection is finished).

Your approach sounds good to me, and I am looking forward to supporting this as an official feature, as well as any input or PRs you might have.

---

As an open question, I do wonder (and plan on testing, when I get a chance) any performance improvements that could be gained by performing both the scene detection and splitting operations simultaneously. This would apply both for approaches, using mkvmerge or ffmpeg, as the scene detection could be performed at the same time the scenes are being outputted. My only concern here is any potential file access issues, so any insight on the issue would be welcome.

Also, this would be a great way to compare how PySceneDetect performs scene cuts versus how I have it implemented in DVR-Scan, which uses an OpenCV VideoWriter for all video output. Although you can select some codecs, the options are extremely limited, and frame accuracy is just as important for that project - thus if this implementation works well, I may adopt this technique for DVR-Scan's video output as well.

[Breakthrough]

Hello all;

This has been added officially to v0.5-beta (you can grab it from the releases page to test it if you would like to provide some feedback; the release version of v0.5 should be out sometime this month).

PySceneDetect now defaults to ffmpeg, but still defaults to copy mode. However, there is a precise mode which defaults to reencode using libx264 providing frame-accurate cuts. The syntax for 'precise' splitting is now:

scenedetect -i input_video detect-content split-video -p

The -p flag is equivalent to setting -f "-c:v libx264 -c:a copy". You can also override the codec settings manually, to set other options with the -f flag, for example:

scenedetect -i input_video detect-content split-video -f "-c:v libx264 -c:a aac"

When using -f, you do need to manually specify both the audio and video codecs to be used that will be passed to the ffmpeg command line. I will leave this issue open until the release of v0.5 so any feedback on the beta version can be provided here.

If ffmpeg is not found, PySceneDetect will fall back to using mkvmerge and not provide frame-accurate cuts. You can also force the use of mkvmerge over ffmpeg by using the -m/--mkv flag with split-video:

scenedetect -i input_video detect-content split-video -m

Thank you all (esp. @martinDRD, @cjbarth, and @Rapsey) for your comments, feedback, suggestions, and help towards making PySceneDetect better 👍