This is the place for all my face based input device projects. Currently there are two major sides:
- Node: this has all the WebRTC stuff for getting webcam in browser as well as doing face detection in browser. The server is node for various reasons (below). Video of google streetview controls
- Processing: using websockets to allow the face tracking to be done in Processing, then piped to the browser. Video of three.js Minecraft controls
Building the node app
Requires node, npm and Chrome. Only tested on a Mac, should work on any computer with a webcam.
git clone firstname.lastname@example.org:paulkaplan/FaceSpace.git cd Faceit npm install node app.js
In Chrome, go to chrome://flags, enable MediaStream flag.
|- / -> Theremin |- /map -> Google street view with face controls |- /pong -> Multiplayer face pong
Troubleshooting / Warning
If you get some permissions errors, try starting Chrome from the terminal with this command: ```/path/to/chrome/app --allow-file-access-from-file
On a Mac, that path is annoyingly long, for reference on my computer it looks like: ```/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --allow-file-access-from-file
Also note: you might want to turn down your speakers, the theremin can catch you by surprise.
Face pong and theremin where created during the hack@uchicago hackathon, april 2012. Map was created shortly thereafter.
Uses now.js for multiplayer action, currently won't really work without two different people logged in. Even then it's pretty broken. Will get back to working on this soon with deterministic physics approach for auto tweening the ball from paddle to paddle, instead of the velocity based approach.
I want to make a blues scale jazz one. Or one where your up and down head motion is the drum beat and side to side is chord progression.
Lots to do here, mostly google street view isn't quite fast enough to keep up. Waiting for them to open up MapGL API. But working on a way to advance down the street in the direction you are facing, have a prototype which is alright I'll put up soon. It is like a real life racing game, pretty sweet actually.
Building the Processing side
This is arguably much easier: 1. grab the openCV Processing library 2. and the websockets library 3. run the sketch, it will start a server which hosts the websocket connection, index.html should be available at localhost:8080.
Feel free to contact Paul if any of this looks interesting to you or you want to get involved. There will be much more to come.