Shot threading and scene detection in TV series
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
cache/shot_similarity add cache for bbt_s01e01 the shot mean RGB histogram of 6x6x6 (216 bins) Mar 5, 2015
data/shot_boundaries add one example videvent for buffy Nov 2, 2016
ext/imhist_nd_fast
initializers
scenes
threading
utilities
visualization
.gitignore
CRAPL-LICENSE.txt
README.md Fixed errors in README docs Dec 8, 2016
first_init.m
run_and_show_threading.m
startup.m

README.md

Shot Threading and Scene Detection

This is an implementation of some basic video processing tools. Currently, it assumes that shot detection has been performed. It also assumes that you have a suitable means of "reading" through video in Matlab, probably best with a frame burst.

The shot threading is further used for scene detection, as proposed in:


StoryGraphs: Visualizing Character Interactions as a Timeline
Makarand Tapaswi, Martin Bäuml, and Rainer Stiefelhagen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
Project page | Paper download | StoryGraphs code


Tested on

Ubuntu 12.04, Ubuntu 14.04, with Matlab versions: R2013a onwards.

First initialization

The first_init.m script will be called on running startup.m the first time. This will ask you to download sample shot frames for one video of The Big Bang Theory. The images have been downscaled by 2. Please follow the other instructions.


Example usage

Initialize the video data. VideoStruct = BBT(1, 1); initParams;

Compute similarity between every shot and N subsequent shots ssim = shot_similarity(VideoStruct, params);

Convert similarity to threads (by finding maximal cliques) [Threads, shot_assigned] = similarity_to_threads(ssim);

Visualize threading using python template engine visualize_threads_via_htmlrender(VideoStruct, Threads, shot_assigned);

Compute scene change locations scene_breaks = dp_scenes(VideoStruct, params);

Visualize scene detection using python template engine visualize_scenes_via_htmlrender(VideoStruct, scene_breaks, params);

Create support for a TV episode by providing the correct interface to VideoStruct (see initializers), and a way to load frames of the video. As an example, the repository includes a few frames of shots of one episode Big Bang Theory.


External toolboxes

  • VLFeat: Vision Libraries in C++ with a nice interface in Matlab
  • MEX/C++ n-d Image Histogram: Thanks to Boris Schauerte, BSD 2-Clause license

Main functions


Visualization

Python Jinja can be used to automatically generate HTML pages to visualize and/or debug both the shot threading and scene detection results. See this blog post for a sneak peak

Changelog

  • 05 Mar 2015: v1.0: Complete working implementation shot threading + scene detection
  • 06 Feb 2015: v0.1: A complete working implementation of shot threading