Feature Modules

Luca Rossetto edited this page Dec 13, 2017 · 2 revisions

THIS PAGE IS CURRENTLY OUT OF DATE AND WILL BE UPDATED SOON

Feature Modules

This page lists the feature modules which cineast uses for feature extraction and retrieval.

Structure

A feature module implements either the Retriever or the Extractor interface, most implement both. The Retriever interface defines the methods used for similarity search while the Extractor interface defines the methods required for feature extraction. Feature modules are located in the ch.unibas.cs.dbis.cineast.core.features package.

Shot descriptors

Each feature module uses a particular shot descriptor as its input. Shot descriptors are aggregates of a subset of information of a shot.

Frame List

The frame list consists of all frames of a shot in chronological order, including the absolute position of the frames within the video.

Average Image

The average image is the pixel-wise color average of all frames of a shot.

Median Image

The median image is the pixel-wise color median of all frames of a shot.

Most Representative Frame

The most representative frame is the one frame of a shot with the smallest pixel-wise color distance to the average frame.

Motion Paths

Motion paths are sparely-tracked trajectories of interest-points.

Duration

The start and end position of a shot with respect to the entire video.

Subtitles

All subtitle elements which would be displayed during the shot.

Modules

The following table lists all feature modules.

Name Type Input Description
AverageColor global color average image Average over all pixels of a shot
AverageColorArp44 local color average image 4x4 ARP partition-wise average color over all pixels of a shot
AverageColorArp44Normalized local color average image Same as AverageColorArp44 but the input image gets normalized first
AverageColorCLD local color average image Color layout descriptor of the average image
AverageColorCLDNormalized local color average image Same as AverageColorCLD but the input image gets normalized first
AverageColorGrid8 local color average image 8x8 grid-wise average color
AverageColorGrid8Normalized local color average image Same as AverageColorGrid8 but the input image gets normalized first
AverageColorRaster local color average image 8x8 color-quantised image, registration is used as similarity measure
AverageFuzzyHist global color average image 15-bin fuzzy color histogram
AverageFuzzyHistNormalized global color average image Same as AverageFuzzyHist but the input image gets normalized first
ChromaGrid8 local color most representative frame 8x8 Grid of average chroma values
CLD local color most representative frame Color Layout descriptor
CLDNormalized local color most representative frame Same as CLD but the input image gets normalized first
Contrast global color most representative frame Conrast value of the frame
DominantColors global color most representative frame Three most dominant colors as determined by k-means clustering
DominantEdgeGrid16 edge most representative frame 16x16 grid of dominant edge direction quantized into 4 directions
DominantEdgeGrid8 edge most representative frame 16x16 grid of dominant edge direction quantized into 4 directions
EdgeArp88 edge most representative frame 8x8 arp partitioned ratio of edge and non-edge pixels
EdgeArp88Full edge frame list same as EdgeArp88 but computed using all frames of a shot
EdgeGrid16 edge most representative frame 16x16 grid partitioned ratio of edge and non-edge pixels
EdgeGrid16Full edge frame list same as EdgeGrid16 but computed using all frames of a shot
EHD edge most representative frame Edge Histogram Descriptor
HueValueVarianceGrid8 local color frame list 8x8 grid partitioned average and variance of hue and value of a shot
MedianColor global color median image median color of a shot
MedianColorArp44 local color median image 4x4 arp partitioned median color
MedianColorARP44Normalized local color median image Same as MedianColorARP44Normalized but the input image gets normalized first
MedianColorGrid8 local color median image 8x8 grid-wise median color
MedianColorGrid8Normalized local color median image Same as MedianColorGrid8but the input image gets normalized first
MedianColorRaster local color median image 8x8 color-quantised image, registration is used as similarity measure
MedianFuzzyHist local color median image 15-bin fuzzy color histogram
MedianFuzzyHistNormalized local color median image Same as MedianFuzzyHistbut the input image gets normalized first
MotionHistogram motion motion paths 8-bin normalized motion histogram
MotionSum motion motion paths sum of total trajectory length
SaturationGrid8 local color most representative frame 8x8 grid-partitioned saturation value
STMP7EH motion frame list STMP7EH motion descriptor
SubDivAverageFuzzyColor local color average image same as AverageFuzzyHist but on a 3x3 grid-partition
SubDivMedianFuzzyColor local color median image same as MedianFuzzyHist but on a 3x3 grid-partition
SubDivMotionHisogram2 motion motion paths Same as MotionHistogram but with a 2x2 grid-partition
SubDivMotionHisogram3 motion motion paths Same as MotionHistogram but with a 3x3 grid-partition
SubDivMotionHisogram4 motion motion paths Same as MotionHistogram but with a 4x4 grid-partition
SubDivMotionHisogram5 motion motion paths Same as MotionHistogram but with a 5x5 grid-partition
SubDivMotionSum2 motion motion paths Same as MotionSum but with a 2x2 grid-partition
SubDivMotionSum3 motion motion paths Same as MotionSum but with a 3x3 grid-partition
SubDivMotionSum4 motion motion paths Same as MotionSum but with a 4x4 grid-partition
SubDivMotionSum5 motion motion paths Same as MotionSum but with a 5x5 grid-partition
SubtitleFulltextSearch text subtitles matches sentence fragments in subtitle elements
SubtitleWordSearch text subtitles matches words independent of order in subtitle elements
Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.