In order to effectively use PySceneDetect, you should become familiar with the basic command line options (especially the detection method commands detect-content and detect-threshold, both of which have an adjustable threshold value option -t / --threshold). Descriptions for all command-line arguments/options can be obtained by running PySceneDetect with the help command, help all, or help [command] (e.g. help detect-content).
There are two main detection methods PySceneDetect uses: threshold (detect-threshold, comparing each frame to a set black level, useful for detecting cuts and fades to/from black), and content (detect-content, compares each frame sequentially looking for changes in content, useful for detecting fast cuts between video scenes, although slower to process). Each mode has slightly different parameters, and is described in detail below.
If the video uses a lot of fast cuts between content, and has no well-defined scene boundaries, you should use the detect-content mode. Use detect-threshold mode if you want to detect scene boundaries using fades/cuts in/out to black.
Once you know what detection mode to use, you can try the parameters recommended below, or generate a statistics file (using the -s / --stats flag) in order to determine the correct paramters - specifically, the proper threshold value. It is always recommended to generate a stats file to speed up subsequent calls to PySceneDetect for the same input video(s).
Unlike threshold mode, content-aware mode looks at the difference between each pair of adjacent frames, triggering a scene break when this difference exceeds the threshold value. A good threshold value to try when using content-aware mode (detect-content) is 30 (-t 30), which is the default, for example the following two commands are equivalent:
scenedetect -i my_video.mp4 -s my_video.stats.csv detect-content
scenedetect -i my_video.mp4 -s my_video.stats.csv detect-content -t 30The optimal threshold can be determined by generating a statsfile (-s) as shown above, opening it with a spreadsheet editor (e.g. Excel), and examining the content_val column. This value should be very small between similar frames, and grow large when a big change in content is noticed (look at the values near frame numbers/times where you know a scene change occurs). The threshold value should be set so that most scenes fall below the threshold value, and scenes where changes occur should exceed the threshold value (thus triggering a scene change).
To automatically split the video based on the detected scenes (will save starting from my_video-Scene-001.mp4, call help split-video for details on changing the output filename format), we add the split-video command at the end:
scenedetect -i my_video.mp4 -s my_video.stats.csv detect-content -t 30 split-videoThreshold-based mode is what most traditional scene detection programs use, which looks at the average intensity of the current frame, triggering a scene break when the intensity falls below the threshold (or crosses back upwards). A good threshold value to try when using threshold mode (detect-threshold) is 12 (-t 12), with a minimum percentage of at least 90% (-m 0.9). Using values less than 8 may cause problems with some videos (especially those encoded at lower quality bitrates).
The optimal threshold can be determined by generating a statsfile (-s), opening it with a spreadsheet editor (e.g. Excel), and examining the delta_rgb column. These values represent the average intensity of the pixels for that particular frame (taken by averaging the R, G, and B values over the whole frame). The threshold value should be set so that the average intensity of most frames in content scenes lie above the threshold value, and scenes where scene changes/breaks occur should fall under the threshold value (thus triggering a scene change).
PySceneDetect can also be used from within other Python programs. This allows you to perform scene detection directly in Python code using a SceneManager, which allows adding specific SceneDetector objects. You can then perform scene detection on frames obtained from a VideoManager object (similar to an OpenCV VideoCapture object but with additional features to facilitate scene detection, like frame-accurate seeking support).
Note: support for live video stream segmentation is supported by the API only currently, not the CLI. See the API documentation on the parameters a VideoManager object constructor takes for details.
The general usage workflow is to determine which detection method and threshold to use (this can even be done iteratively), using these values to create a SceneDetector object, the type of which depends on the detection method you want to use (e.g. ThresholdDetector, ContentDetector). These detectors are then added to a SceneManager class, with optionally a StatsManager to cache frame metrics so subsequent scene detection runs are much faster (and can be saved/loaded to/from disk). Finally, an open VideoManager object can be passed to the SceneManager.detect_scenes() method, which returns the number of frames processed.
The following shows the contents of the api_test.py file included with the PySceneDetect source code, which provides an example as to the general usage of the PySceneDetect Python API:
from __future__ import print_function
import os
import scenedetect
from scenedetect.video_manager import VideoManager
from scenedetect.scene_manager import SceneManager
from scenedetect.frame_timecode import FrameTimecode
from scenedetect.stats_manager import StatsManager
from scenedetect.detectors import ContentDetector
STATS_FILE_PATH = 'api_test_statsfile.csv'
def main():
print("Running PySceneDetect API test...")
print("PySceneDetect version being used: %s" % str(scenedetect.__version__))
# Create a video_manager point to video file testvideo.mp4. Note that multiple
# videos can be appended by simply specifying more file paths in the list
# passed to the VideoManager constructor. Note that appending multiple videos
# requires that they all have the same frame size, and optionally, framerate.
video_manager = VideoManager(['testvideo.mp4'])
stats_manager = StatsManager()
scene_manager = SceneManager(stats_manager)
# Add ContentDetector algorithm (constructor takes detector options like threshold).
scene_manager.add_detector(ContentDetector())
base_timecode = video_manager.get_base_timecode()
try:
# If stats file exists, load it.
if os.path.exists(STATS_FILE_PATH):
# Read stats from CSV file opened in read mode:
with open(STATS_FILE_PATH, 'r') as stats_file:
stats_manager.load_from_csv(stats_file, base_timecode)
start_time = base_timecode + 20 # 00:00:00.667
end_time = base_timecode + 20.0 # 00:00:20.000
# Set video_manager duration to read frames from 00:00:00 to 00:00:20.
video_manager.set_duration(start_time=start_time, end_time=end_time)
# Set downscale factor to improve processing speed.
video_manager.set_downscale_factor()
# Start video_manager.
video_manager.start()
# Perform scene detection on video_manager.
scene_manager.detect_scenes(frame_source=video_manager,
start_time=start_time)
# Obtain list of detected scenes.
scene_list = scene_manager.get_scene_list(base_timecode)
# Like FrameTimecodes, each scene in the scene_list can be sorted if the
# list of scenes becomes unsorted.
print('List of scenes obtained:')
for i, scene in enumerate(scene_list):
print(' Scene %2d: Start %s / Frame %d, End %s / Frame %d' % (
i+1,
scene[0].get_timecode(), scene[0].get_frames(),
scene[1].get_timecode(), scene[1].get_frames(),))
# We only write to the stats file if a save is required:
if stats_manager.is_save_required():
with open(STATS_FILE_PATH, 'w') as stats_file:
stats_manager.save_to_csv(stats_file, base_timecode)
finally:
video_manager.release()
if __name__ == "__main__":
main()The scene list returned by the SceneManager.get_scene_list(...) method consists of the start and (one past) the end frame of each scene, in the form of a FrameTimecode object. Each FrameTimecode can be converted to the appropriate working/output format via the get_timecode(), get_frames(), or get_sceonds() methods as shown above; see the API documentation for FrameTimecode objects for details.
Licensed under BSD 2-Clause (see the LICENSE file for details).
Copyright (C) 2012-2018 Brandon Castellano. All rights reserved.