Strike(this.Pose) is a web app that uses a webcam and Microsoft Cognitive Services APIs to test your ability to match a series of facial expressions. Test it at https://strikethispose.azurewebsites.net. (Note: getUserMedia API is not compatible with Safari on OS X and iOS.)
How it works
This web application uses the
getUserMedia API in the browser to capture a live video feed from a webcam. When the user clicks the "Snap photo" button, a still image from the video is written to a
<canvas>. The image is then sent as a data URL to two Cognitive Services APIs: The Face API and the Emotion API. The existing photo being matched against is sent to these APIs as well. Finally, selected results are compared and a score is calculated by way of a Very Complicated Algorithm*. There are a few other technologies working behind the scenes, including websockets and Node.js.
D.I.Y.: Deploy It Yourself!
You can clone this repo to your local machine and run it locally. Or, you can fork it and have it autodeploy to an Azure Web App.
- Clone this repo to your local machine (
git clone https://github.com/hxlnt/strikethispose).
process.env.emokeywith your own (free) Cognitive Services API keys. You can put these keys in a separate env file or right in
cdinto the repo and run
node app.js. This will start a server at http://localhost:3002.
OR: Deploy to Azure
- Fork this repo.
- Connect your fork to an Azure Web App. See this page for step-by-step instructions.
- Add your own (free) Cognitive Services API keys under your Web App's Application Settings. Be sure to call the keys
emokey, respectively. In the same Application Settings menu, switch websockets on.
Once you have the app running, open up
js/script.js and replace the images in
imgarray with your own photos! That way, you can challenge your friends to make facial expressions that mimic you. Or pop stars. Or actors. Or whoever. Do note that if you want to use local paths here instead of remote links, you'll need to replace both instances of
This app was built as a quick up-and-running prototype, and the code has some less-than-best practices :). Here are a few outstanding issues. Feel free to submit an issue/PR related to these or other issues if you're so inclined.
- Improve error handling
- Improve browser resize behavior
- Improve mobile phone experience
- Improve code readability (e.g., mix of similar JQuery/not-Jquery methods is fine but hurts readability)
- Replace hacky things like overwriting contents of
More demos, please!
I've got another repository that demonstrates a very useful feature of the Face API: making DEAL WITH IT memes.
Here are Microsoft's official SDKs and samples for Cognitive Services.
gl,hf! xoxox rae