-
Notifications
You must be signed in to change notification settings - Fork 9
/
module-rtc-videoproc.html
173 lines (161 loc) · 11.5 KB
/
module-rtc-videoproc.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="format-detection" content="telephone=no">
<meta name="viewport" content="user-scalable=no, initial-scale=1, maximum-scale=1, minimum-scale=1, width=device-width">
<title>rtc.io</title>
<link rel="stylesheet" type="text/css" href="css/reset.css">
<link rel="stylesheet" type="text/css" href="css/main.css">
<!-- responsive -->
<link rel="stylesheet" media="screen and (max-width: 960px)" href="css/tablet.css">
<link rel="stylesheet" media="screen and (max-width: 710px)" href="css/phone.css">
<link rel="stylesheet" type="text/css" href="fonts/source-sans/stylesheet.css">
<link rel="stylesheet" type="text/css" href="css/code.css">
</head>
<body>
<a class="scroll-point pt-top" name="top"></a>
<header>
<a href="https://github.com/rtc-io/rtc-videoproc"><img class="fork" src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png" alt="Fork me on GitHub"></a>
<a class="scroll-point pt-top" name="top"></a>
<div class="site">
<div class="mascot">
<img src="images/artsio.png">
</div>
<div class="logo" data-subtext="OpenSource WebRTC">
<a href="index.html">rtc.io</a>
</div>
<nav>
<ul>
<li><a href="index.html">About</a></li>
<li><a href="tutorials.html">Tutorials</a></li>
<li><a href="demos.html">Demos</a></li>
<li><a href="modules.html">Modules</a></li>
</ul>
</nav>
</div>
<div class="shadow"></div>
</header>
<div class="main" role="content"><h1 id="rtc-videoproc">rtc-videoproc</h1>
<p>This is a small helper module that allows you to substitute a video
element with a canvas element. This can be useful when you want to
do pixel manipulation of the rendered images, or in situations when
a video element does not behave as you expect.</p>
<p><a href="https://nodei.co/npm/rtc-videoproc/"><img src="https://nodei.co/npm/rtc-videoproc.png" alt="NPM"></a></p>
<p><a href="https://travis-ci.org/rtc-io/rtc-videoproc"><img src="https://img.shields.io/travis/rtc-io/rtc-videoproc.svg?branch=master" alt="Build Status"></a> <a href="https://github.com/dominictarr/stability#unstable"><img src="https://img.shields.io/badge/stability-unstable-yellowgreen.svg" alt="unstable"></a> </p>
<h2 id="example-usage">Example Usage</h2>
<div class="highlight"><pre><span class="kd">var</span> <span class="nx">crel</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'crel'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">videoproc</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'rtc-videoproc'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">video</span> <span class="o">=</span> <span class="nx">crel</span><span class="p">(</span><span class="s1">'video'</span><span class="p">,</span> <span class="p">{</span> <span class="nx">src</span><span class="o">:</span> <span class="s1">'../test/assets/tennis.webm'</span><span class="p">,</span> <span class="nx">autoplay</span><span class="o">:</span> <span class="kc">true</span> <span class="p">});</span>
<span class="kd">var</span> <span class="nx">canvas</span> <span class="o">=</span> <span class="nx">crel</span><span class="p">(</span><span class="s1">'canvas'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">processor</span> <span class="o">=</span> <span class="nx">videoproc</span><span class="p">(</span><span class="nx">video</span><span class="p">,</span> <span class="nx">canvas</span><span class="p">,</span> <span class="p">{</span>
<span class="nx">filter</span><span class="o">:</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'rtc-filter-grayscale'</span><span class="p">)</span>
<span class="p">});</span>
<span class="nb">document</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">appendChild</span><span class="p">(</span><span class="nx">canvas</span><span class="p">);</span>
</pre></div>
<h2 id="using-the-processing-pipeline">Using the Processing Pipeline</h2>
<p>A processing pipeline has been included to assist with
manipulating the canvas on the fly. To specify the filters to be used
in the processing pipeline, this is done in the options accepted by
videoproc. Either specifying an array of filters with the <code>filters</code> option
or a single filter function with the <code>filter</code> option is fine. If you use
both then the individual filter will be added filter list and used in
series.</p>
<div class="highlight"><pre><span class="nx">videoproc</span><span class="p">(</span><span class="nx">srcVideo</span><span class="p">,</span> <span class="nx">targetCanvas</span><span class="p">,</span> <span class="p">{</span>
<span class="nx">filters</span><span class="o">:</span> <span class="p">[</span>
<span class="nx">require</span><span class="p">(</span><span class="s1">'rtc-filter-grayscale'</span><span class="p">),</span>
<span class="nx">myCustomFilterFunction</span>
<span class="p">]</span>
<span class="p">});</span>
</pre></div>
<h2 id="writing-a-filter-function">Writing a Filter Function</h2>
<p>Writing a filter function is very simple, and they use the following
function signature:</p>
<div class="highlight"><pre><span class="kd">function</span> <span class="nx">filter</span><span class="p">(</span><span class="nx">imageData</span><span class="p">,</span> <span class="nx">tick</span><span class="p">)</span> <span class="p">{</span>
<span class="p">}</span>
</pre></div>
<p>The <code>imageData</code> arg is an
<a href="http://www.w3.org/TR/2dcontext/#imagedata">ImageData</a>, and the <code>tick</code>
argument refers to the tick that has been captured as part of the capture
loop (be aware that it could be a high resolution timer value if rAF is
being used).</p>
<p>If you are writing an analysis filter, then simply do what you need to do
and exit the function. If you have written a filter that modifies the pixel
data and you want this drawn back to the canvas then your <strong>filter must
return <code>true</code></strong> to tell <code>rtc-videoproc</code> that it should draw the imageData
back to the canvas.</p>
<h2 id="listening-for-custom-frame-events">Listening for custom <code>frame</code> events</h2>
<p>In addition to providing the opportunity to analyse and modify pixel data
the <code>rtc-videoproc</code> module also provides the a custom <code>frame</code> event for
detecting when a new frame has been drawn to the canvas.</p>
<p>A simple example can be found below:</p>
<div class="highlight"><pre><span class="kd">var</span> <span class="nx">videoproc</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'rtc-videoproc'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">crel</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'crel'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">video</span> <span class="o">=</span> <span class="nx">crel</span><span class="p">(</span><span class="s1">'video'</span><span class="p">,</span> <span class="p">{</span> <span class="nx">src</span><span class="o">:</span> <span class="s1">'../test/assets/tennis.webm'</span><span class="p">,</span> <span class="nx">autoplay</span><span class="o">:</span> <span class="kc">true</span> <span class="p">});</span>
<span class="c1">// set up the video processing pipeline</span>
<span class="nx">videoproc</span><span class="p">(</span><span class="nx">video</span><span class="p">).</span><span class="nx">on</span><span class="p">(</span><span class="s1">'frame'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">tick</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'captured frame at: '</span> <span class="o">+</span> <span class="nx">tick</span><span class="p">);</span>
<span class="p">});</span>
<span class="c1">// add the canvas to the dom</span>
<span class="nb">document</span><span class="p">.</span><span class="nx">body</span><span class="p">.</span><span class="nx">appendChild</span><span class="p">(</span><span class="nx">video</span><span class="p">);</span>
</pre></div>
<p>NOTE: The <code>frame</code> event occurs after the filter pipeline has been run and
and the imageData may have been modified from the original video frame.</p>
<h2 id="a-note-with-regards-to-cpu-usage">A Note with Regards to CPU Usage</h2>
<p>By default rtc-videoproc will draw at 25fps but this can be modified to capture
at a lower frame rate for slower devices, or increased if you have a
machine with plenty of grunt.</p>
<h2 id="reference">Reference</h2>
<h3 id="videoproc-src-target-opts-">videoproc(src, target, opts?)</h3>
<p>Create (or patch) a <code><canvas></code> element that will receive the video images
from a video element. The following options are supported.</p>
<ul>
<li><p><code>canvas</code> - the canvas to draw video data to. If not supplied a new
canvas element will be created.</p>
</li>
<li><p><code>video</code> - the video element that will be used as the source of the video.
If not supplied a new <code><video></code> element will be created.</p>
</li>
<li><p><code>fps</code> - the redraw rate of the fake video (default = 25)</p>
</li>
<li><p><code>greedy</code> - Specify <code>greedy: true</code> if you want the videoproc module to run
it's capture loop using setTimeout rather than <code>requestAnimationFrame</code>.
Doing this will mean you application will continue to capture and process
frames even when it's tab / window becomes inactive. This is usually the
desired behaviour with video conferencing applications.</p>
</li>
</ul>
<h2 id="license-s-">License(s)</h2>
<h3 id="apache-2-0">Apache 2.0</h3>
<p>Copyright 2014 National ICT Australia Limited (NICTA)</p>
<p> Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at</p>
<pre><code> http://www.apache.org/licenses/LICENSE-2.0</code></pre>
<p> Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.</p>
</div>
<footer>
<p>
<a href="http://nicta.com.au">
<img src="images/nicta-logo.gif" alt="NICTA logo">
</a>© NICTA 2013 - 2014
</p>
<p class="license">Project source code is licensed under the <a href="https://github.com/rtc-io/rtc/blob/master/LICENSE">Apache 2.0</a>.</p>
<a class="closing" href="#top"></a>
</footer>
</body>
<script src="js/app.js"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-26567546-2', 'rtc.io');
ga('send', 'pageview');
</script>
</html>