An Audio-Visual Experiment in the Browser
C C++ JavaScript
Pull request Compare This branch is 27 commits behind mozilla:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Rainbow is an early prototype that provides video and audio capture
capabilities to web pages. It is currently distributed as a highly experimental
Firefox addon.

Rainbow uses a bunch of different backends depending on the platform to
capture audio and video. The ogg, theora and vorbis libraries from xiph are
used to encode them. WebM support is on our radar.

Mac: libvidcap for Video, portaudio for Audio
Windows: DirectShow for Video, and WinMM for Audio
Linux: Video not supported, ALSA for Audio

We would ultimately not want to depend on any external libraries. QTKit
backends for Video and Audio on the Mac, as well as a V4L backend for Video on
Linux are under development.

Quick testing
We have setup test pages for Rainbow on
which you can visit after you install the add-on. The domain has already been
added to the default preference 'extensions.rainbow.allowedDomains', but
if you are upgrading from an older version, please add ''
manually to the preference via about:config.

The JS API offers the capability to record multiplexed audio and video files
to disk:, ctx);

where 'params' is a JS Object, which may contain the following properties.
The types and default values for each property are in brackets:

    audio:(boolean, true),
    video:(boolean, true),
    width:(int, 640),
    height:(int, 480),
    rate:(int, 22050),
    channels:(int, 1),
    quality:(float, 0.4)

and 'ctx' is the 2D context of a canvas on which a live preview (if video was
requested) will be drawn. 'ctx' may be null in case you do not want a preview.
The record call will return an object, on which you can call stop() to end
recording. It will return a DOM File object which you may then upload.
Please see content/example.html for sample usage of the API.

Rainbow 0.3 also offer support for recording from a canvas as source (instead
of a webcam). The canvas will be sampled the rate of 30 frames per second. You
can do this by specifying the 'source' property of the params object to be
'true'. This will treat the 'ctx' argument you pass to the recordToFile function
as the source of video rather than the destinatin of a live preview. Please
see content/example_canvas.html for an example of such usage.

For the really bold, Rainbow 0.3 also comes with experimental support for 
streaming to an Icecast server. See content/example_stream.html to see how this
is done. You will have to configure your Icecast server to accept the username /
password combination 'source:rainbow'. We have dropped support for writing
streams to a websocket (since they will not be enabled by default in Firefox 4).

Temporary Files
Rainbow does not correctly clean up files generated by it. You must manually
remove recorded OGG files from your disk. These files may be found in:

Mac: ~/Library/Caches/TemporaryItems/
Windows: C:\Users\<user>\AppData\Local\Temp\
Linux: /tmp

We are working on an API to ensure that these files get cleaned up once
the web application that requested their creation is done with them.

Common Errors
"Error: Permission denied for <http://localhost> to call method
    UnnamedClass.toString on <>.
 Error: uncaught exception: unknown (can't convert to string)"

This usually means that a device to record audio or video was not found. If you
look at the command line (stderr) you will find a clearer error message. We need
to propagate this type of error correctly to JS. Verify that your camera and
microphone are working with a third party application and re-run the examples
with either audio or video disabled.

"Error: window.navigator.service is undefined"

This can mean one of two things:

- You tried to access the API from a webpage hosted on a restricted domain.
  By default, Rainbow will only allow access to web pages served from localhost,
  but you may change that by adding more domains to the
  preference through about:config. The value is simply a JSON array of domains.

- The binary component itself failed to load. This may because you are using
  an older version of Firefox, or that you are running the nightly on an
  unsupported platform (see below). If this is the case the error will be
  preceded by something like:

"Error: Components.classes[';1'] is undefined"

Platform Support
On Linux, we do not support video recording.

On Windows, some DV cameras may result in garbled video.

On Mac (Snow Leopard), 64-bit versions of Minefield are not supported. You
may run your Minefield nightly in 32-bit mode like so:
$ arch -i386 /Applications/

Video Playback 
Rainbow 0.1 produced incorrectly encoded OGG files, the audio component of
which could not be played by Firefox. Simply upgrade to Rainbow 0.2 to fix
the issue.

If you wish to build the binary components yourself, fetch a recent build of
the Gecko2 SDK from:
After extracting, set the MOZSDKDIR environment variable to point to it and
simply run `make'.

Precompiled versions of the dependencies are already included for
convenience, however if you wish to include custom versions of libvidcap,
libportaudio, libogg, libvorbis or libtheora; place them in the appropriate
directory in lib/ and the build system will automatically link with them.

You may set debug=1 to create a version of the XPCOM component with debugging