-
Notifications
You must be signed in to change notification settings - Fork 59
Effects
Effects allow you to apply full frame pixel manipulations to a MediaSource before it is rendered to the final output. Effects take the form of a GLSL vertex and/or fragment shader. By default every MediaSource has a simple effect applied to it which renders the MediaSource to cover the full frame. There's currently support for passing in a limited range of parameters to an effect which can be manipulated at runtime.
- Chroma Keying
- Scaling & Offsetting
- Monochroming
- In/Out Fades
- Only one effect per MediaSource.
- A MediaSource can have it's effect switched while it is playing.
- An effects parameters can be updated live.
- Effects only operate on a MediaSource in isolation, although they can have multiple inputs for independent runs on many MediaSources
var playlist = {
"tracks":[
[{type:"video", src:"assets/clip1.mp4", start:0, duration:2, id:"clip1"}]
]
"effects":{
"custom-effect-clip1": { // A unique name used to identify the effect
inputs : ["clip1"], // A list of ID's of MediaSourceReferences which this effect will operate on
effect : { //
"id": "custom-effect", // A unique ID used to cache the shader program internally (future versions will remove this)
"vertexShader": "\ // GLSL code for the fragment shader, this shader draws a full frame image
uniform float progress;\ // All shaders have a progress parameter passed in, this varies from 0.0 to 1.0 of the duration the MediaSource the shader is acting on.
uniform float duration;\ // All shaders have a duration parameter passed in, this is the duration of a MediaSourceReference.
attribute vec2 a_position;\
attribute vec2 a_texCoord;\
varying vec2 v_texCoord;\
varying float v_progress;\
varying float v_duration;\
void main() {\
v_progress = progress;\ // This makes the duration parameter accessible in the fragment shader
v_duration = duration;\ // This makes the duration parameter accessible in the fragment shader
gl_Position = vec4(vec2(2.0,2.0)*a_position-vec2(1.0, 1.0), 0.0, 1.0);\
v_texCoord = a_texCoord;\
}",
"fragementShader" : "\ // GLSL code for the fragment shader, this shader draws each pixel with no modification
precision mediump float;\
uniform sampler2D u_image;\
varying vec2 v_texCoord;\
varying float v_progress;\
varying float v_duration;\
void main(){\
gl_FragColor = texture2D(u_image, v_texCoord);\
}",
}
}
}
}This sections covers some common effect operations and how you'd go about using them.
- VideoCompositor.Effect.OFFSETSCALE
- VideoCompositor.Effect.MONOCHROME
- VideoCompositor.Effect.SEPIA
- VideoCompositor.Effect.BITCRUNCH
- VideoCompositor.Effect.GREENSCREEN
- VideoCompositor.Effect.FADEINOUT
- VideoCompositor.Effect.FADEINOUT1SEC
- VideoCompositor.Effect.FADEINOUT2SEC
- VideoCompositor.Effect.FADEIN1SEC
- VideoCompositor.Effect.FADEIN2SEC
- VideoCompositor.Effect.FADEOUT1SEC
- VideoCompositor.Effect.FADEOUT2SEC
Crossfading is a common operation, and although we can't have more than one input per effect we can simulate a crossfade by fading one media source down while fading another up. The following demonstrates a 1 second crossfade between two videos.
var playlist = {
"tracks":[
[{type:"video", src:"assets/clip1.mp4", start:0, duration:2, id:"clip1"},{type:"video", src:"assets/clip2.mp4", start:1, duration:2, id:"clip2"}]
]
"effects":{
"fade-out-clip1": {
inputs : ["clip1"],
VideoCompositor.Effects.FADEOUT1SEC
},
"fade-in-clip2" : {
inputs : ["clip2"],
VideoCompositor.Effects.FADEIN1SEC
}
}
}Greenscreening is a surprisingly complex area. In essence it involves taking a color of a specific chrominance and converting it to a transparent alpha value. This can let you do things such as superimpose a weather presenter on-top of a dynamically rendered weather graphic. More complex implementations can feather the edges between alpha and non alpha regions and do perfom hole filling. The following example use the naive internal greenscreening algorithm to composite a presenter over a background image.
var playlist = {
"tracks":[
[{type:"video", src:"assets/presenter.mp4", start:0, duration:10, id:"presenter"}],
[{type:"video", src:"assets/background.mp4", start:0, duration:10, id:"background"}]
]
"effects":{
"greenscreen-clip1": {
inputs : ["presenter"],
VideoCompositor.Effects.GREENSCREEN
},
}
}Parameters can be passed into shaders to allow them to modify some aspect of the rendering. All parameters can be updated live as the VideoCompositor is playing. Examples of when you might want to include passing in of parameters is altering the duration of a crossfade, or dynamically cropping/scaling a video based on user input.
All parameters that can be passed into shaders have a default value, this is used if one is not explicitly set.
The following demonstrate passing in parameters to adjust the in and out times of the build-in crossfade shader.
var playlist = {
"tracks":[
[{type:"video", src:"assets/viceo.mp4", start:0, duration:10, id:"presenter"}],
]
"effects":{
"greenscreen-clip1": {
inputs : ["presenter"],
VideoCompositor.Effects.FADEINOUT
parameters:{
inTime: 2.5,
outTime: 3.14,
}
},
}
}The following playlist demonstrates a simple custom bit-crunching shader effect:
var playlist = {
"tracks":[
[{type:"video", src:"assets/title.mp4", start:0, duration:2, id:"title"},{type:"video", src:"assets/clip1.mp4", start:2, duration:8, id:"clip1"}]
]
"effects":{
"bitcrunch-filter":{
"id":"bitcrunch-filter", //A Unique ID used inside the VideoCompositor to cache shader programs.
"fragmentShader":"\ //Specify the fragment shader.
precision mediump float;\
uniform sampler2D u_image;\
varying vec2 v_texCoord;\
void main(){\
vec4 pixel = texture2D(u_image, v_texCoord);\
pixel = floor(pixel*vec4(8.0,8.0,8.0,8.0));\
pixel = pixel/vec4(8.0,8.0,8.0,8.0);\
gl_FragColor = pixel*vec4(1.0,1.0,1.0,1.0);\
}"
}
}
}