Face interface!?
A short story about a project I did a few weeks ago with my co-student: Mohamad Al Ghorani. (10 min read)
Author: Wouter van der Heijde - June 2020
Dramatic music starts playing
I remember this week so well, it was the last week before COVID-19 shook down our existence and life as we known it. It was raining all week but it never stopped me from reaching the public library of Amsterdam. To work on an interactive project to search books in their data-set. Our prototype was designed to help children make a selection of topics based on their smile. While the week was really intense we really had so much fun working on this project.
So the idea about this project was about helping children to find a book by topic they would like so they can write a essay about it. But how do you know when a kid likes a topic? "Ah, how a bout a smile? If I like something I see, I usually start smiling!" So this could be a perfect way to find a topic that fits the children that don't know where to write about. And the concept was born: Choosing topics based on a smile, and present a list of books that match with that topic. So we would present images to the children about: Animals, Sports, History etc. And then when a user smiles they choose that topic and maybe even a sub-topic after that with eventually a list of books.
We discovered an opensource face recognition package that could be installed on a local server (no internet required). So we build ourselves a node.js server and set up the package to work with the our laptops camera. We aligned the Books-API from OBA (openbare bibliotheek van Amsterdam) and predefined a few topics in our prototype.
- node.js
- face recognition package
- Book-API from OBA
- Modern browsers like: Google Chrome or Firefox
- Java-script turned on
We intended to design this application to be run on a module in the public space of the library so people could freely interact with it. Therefore privacy matters allot. The visitors of the library didn't choose to be continuously recorded while searching for books on the computer. But with this package no videos are stored at all or analyzed for other purposes then finding books. Because the face recognition application runs completely local all the data is safe. There are no video's recorded and the only requests from our server are sent to the books-API from OBA which on their turn serve a list of relevant books.
- choosing topics with your smile
- presenting a list of books based on your choice
- showing results fitting to the users age (kids get kid books, adults get adult books)
- showing result based on emotion (more a gimmick then a feature)
- adjusting styling based on user age (children get a more playful design: different colors)
- people with motor impairments are also able to find books without using their hands.
- COVID-19 restraints, force us to design hands-free ways of interactions
However we only build this project in one week, it was more an experimental project, it really opened our eyes about new interaction methods. We both had never seen an application like this where a user was able to interact with a computer based on their face-expressions. We were both really curious what these technology's can bring us in te near future. Imagine interacting with your house, car or other devices with your face. This giving a whole new meaning to the definition inter-face :).
Curious about the face recognition software we used? Follow this link to their documentation