Skip to content

Playlists

Jump to bottom
Matthew Shotton edited this page Oct 28, 2015 · 7 revisions

Introduction

var playlist = {
    "tracks":[
        [{type:"image", src:"assets/aston.png", start:0, duration:10, id:"aston"}],
        [{type:"video", src:"assets/title.mp4", start:0, duration:2, id:"title"},{type:"video", src:"assets/title.mp4", start:2, duration:8, id:"clip1"}]
    ]
}

The VideoCompositor plays playlists. These are JavaScript objects which represent a sequence of MediaSourceReferences to be played. The MediaSourceReference can point to one of a number of predefined media types, and you can also create custom MediaSources to be rendered into the final composition (so long as they present a set of expected functions).

Tracks are a list of MediaSourceRefernces. The MediaSourceRefernces on a track are played in sequence and multiple tracks can be playing at the same time. The rules for how these are rendered in the final output can be seen in the Rendering Behaviours section of this wiki.

Format of a MediaSourceReference

A MediaSourceReference is used to instantiate a MediaSource internally in the VideoCompositor. It is a javascript object of the following format:

{
    id:"title",             //A unique ID used to identify the created MediaSource
    type:"video",           //A type can be either 'canvas', 'video', or 'image'
    src:"assets/title.mp4", //A URL to load the media from, only works for 'image' and 'video' types.  (See Element vs Source below) 
    start:0,                //The time at which to start playing the created MediaSource 
    sourceStart: 10         //Only applicable to type 'video', the time from within the source file to start playing.
    duration:2              //How long to display the created MediaSource for.
}

Effects

For full documentation please see the Effects page.

Effects are GLSL shaders that get applied to the MediaSource as it is being rendered onto the final output canvas. There are a number of predefined effects available as static properties of the VideoCompositor class.

The following example applies a predefined green-screen filter to the MediaSource with ID "clip1".

var playlist = {
    "tracks":[
        [{type:"video", src:"assets/title.mp4", start:0, duration:2, id:"title"},{type:"video", src:"assets/clip1.mp4", start:2, duration:8, id:"clip1"}]
    ]
    "effects":{
        "greenscreen-filter":{  //This is a user specified label for the effect, it must be unique
            "inputs":["clip1"], //This is the MediaSource to act upon, this can be a list to run multiple MediaSources through the same shaders.
            "effect":VideoCompositor.Effects.GREENSCREEN //This is one of the predefined shaders available from the VideoCompositor.
        }
    }
}

Current Limitations

  • Only one effect per track.
  • API is still in-flux.
  • No more than one texture input per effect shader.

Dynamically Appending To Playlists

MediaSourceReferences can be dynamically appended to Playlists and the VideoCompositor should be able to handle dynamically loading the content in. Due to the VideoCompositors look-ahead loading mechanism the sooner MediaSourceReferences can be appended to the playlist the better.

Track - Rules

  • MediaSourceReferences on a single track must never overlap.
  • MediaSourceReferences on a single track must be ordered in the sequence in which they are to be played.

MediaSourceReference - Rules

  • Must have a globally unique ID
  • Must have EITHER an element passed in OR a src property, never both.
  • If an element is passed in it is the responsibility of the user to handle its resources (i.e cleanly bring it down).

MediaSourceReference - Element Vs Src

In most instances a MediaSourceReference can either be created with a src attribute passed in OR an element attribute, never both. If a src attribute is passed in it is implied that the VideoCompositor will handle creating/destroying the underlying element. If the element is passed in it is implied that the user is responsible for handling the safe destruction of the element.

Most of the time when playing back videos or images MediaSourceReferences will be created with by passing in a src property.

You might want to create a MediaSourceReference with an element property when the element you are rendering might need some form of external control or input, or require custom set-up within the element. For example you may want to composite in a WebGL rendering of a globe and add a marker for the users location onto the globe. You will probably want to instantiate the WebGL context yourself and set up your scene and callbacks for position changes before handing it to the composition engine.

MediaSource vs MediaSourceReference

A MediaSource is an object which is (usually) created internally by the VideoCompositor to wrap a range of media elements in a set of base functionality (play, pause, seek, loading etc.). These are created dynamically as the playlist is being played by the VideoCompositor, and cleanly destroyed once they are done with (textures released etc.). A MediaSourceReference is a description of a MediaSource which lives on a timeline, it is the data used by the VideoCompositor to create/destroy a MediaSource object.

MediaSource Listner API

In order for a custom media source to be played by the VideoCompositor it must support the following methods on an instance. play() pause() seek() load() destroy() render()

Clone this wiki locally