Skip to content
Image tracking with getUserMedia
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


Video stream object tracking in the browser.

recognising a lemon


This recognises objects from a getUserMedia video stream - we built it for a web-based pictionary game at our christmas party.

The approach taken is:

  1. convert the image into HSV
  2. perform thresholding on the three channels
  3. find the average x/y coordinate of matching pixels

This is quite a basic approach to object recognition, if you want to try and recognise more complex objects you might want to take a look at js-objectdetect, or skip the browser and go stright to Also, if you have suggestions of how to make this more robust/efficient - please do log an issue or ping benjaminbenben.

Most of the processing is done in a web worker, so it shouldn't impact too much on other page interactions. There is also an optimisation that only checks the region near the last observed point - which speeds things up quite a bit.


<div id="target"></div>
<button id="calibrate_button"></button>

<script src="js/bauble.js"></script>
  var bauble = new Bauble({worker:'js/bauble-worker.js'})
      .on('point', function(x,y){
        // do something awesome!!

  // click #calibrate_button to mark as calibrated
  calibrate_button.onclick = function(){


There is an offscreen canvas that is drawn over the top of every video frame, this can be handy for overlaying visuals on the video.

.on('point', function(x,y){


There is a pretty basic example at @benjaminbenben's presentation at #ogn34. More are on the way.

You can’t perform that action at this time.