Skip to content
A wrapper library around web-based computer vision models for the purpose of interacting with the web handsfree
Vue JavaScript HTML CSS
Branch: master
Clone or download
ozramos Merge pull request #21 from handsfreejs/dev
Adds the 1st person route (not finished)
Latest commit 45d7679 Nov 14, 2019

~ Presenting ~

with support from, the STUDIO at CMU, the School of AI and you!



A wrapper library around web-based computer vision models for the purpose of interacting with the web handsfree

Powered by:


<!DOCTYPE html>
  <!-- Require dependencies, which adds Handsfree to global namespace -->
  <script src=""></script>
    // Create a new instance. Use one instance for each camera
    const handsfree = new Handsfree({})

    // Create a simple plugin that displays pointer values on every frame
    // When using only 1 instance, handsfree === context
    Handsfree.use('consoleLogger', (pointer, context) => {
      console.log(pointer, context.head.rotation, context.head.morphs)

    // Start tracking

Learn how to use Handsfree.js

Config · Methods · Properties · The Head · Plugins · Events · Classes


The rest of this document is for running and the Handsfree.js library locally.

👆 See the above wiki links for how to actually use Handsfree.js in your own app 👆

Local Development

A note about this codebase

This codebase is currently broken into two parts:

  • The library itself, located in /src/assets/handsfree/handsfree.js
  •, which is everything else

This really should be two separate repositories, but for now just know that the library itself starts in /src/assets/handsfree/handsfree.js

To run this project locally you'll need NodeJS and the Yarn package manager.

After downloading this project repo, you'll then need to install dependencies by running yarn in the project's root directory. Then you'll have the following commands available:

# Start a local dev environment on localhost:8080
yarn start

# Build for production into /dist/sandbox/
yarn build

# Deploy to
yarn deploy

# Create handsfree.js in /dist/handsfreejs/
yarn bundle

Deploy Script

Running yarn deploy will commit everything inside of /dist to the gh-pages branch of the repository set in package.json:deploy.repo using the package.json:deploy.url custom domain. This lets you quickly deploy this repository to GitHub Pages!

Creating the handsfree.js library

When you run yarn start, yarn build, or yarn deploy what you're actually doing is running or building the development environment. To create a single handsfree.js script for use within your own projects, do the following:

  • Install Parcel on your system globally with: yarn global add parcel-bundler
  • Run npm bundle
  • The files will be built into /dist/handsfree/

License & Attributions

Handsfree.js is available for free and commercial use under Apache 2.0.


  • Jeeliz Weboji (Apache License 2.0) - Face and head pose estimation


Special Thanks

A very special thanks goes out to @Golan for inviting me out to his studio, The STUDIO for Creative Inquiry at Carnegie Mellon during the Spring of 2019. It was during this residency that I was encouraged to begin integrating Handsfree.js into different libraries and where I had a chance to use Handsfree.js with a real UR5 robot!

Another special thanks goes out to @AnilDash for sponsoring me during Winter 2018. Also a thank you to The School of AI for the 2018 Fellowship. And a very special thanks to Jess Holbrook from Google Pair for driving all the way (way) out to meet me and helping kickstart this all with a new computer!

Thanks also to everyone who's supported me on Patreon, GoFundMe, and through Twitter over the months (and almost years!). And thanks everyone else for believing in this project 👋

You can’t perform that action at this time.