Face interface!?
A short story about a project I did a few weeks ago with my co-student: Mohamad Al Ghorani. (10 min read)
Author: Wouter van der Heijde - June 2020
Dramatic music starts playing
I remember this week so well, it was the last week before COVID-19 shook down our existence and life as we knew it. It was raining all week but it never stopped me from reaching the public library of Amsterdam to work on an interactive project to search books in their data-set. Our prototype was designed to help children finding a selection of topics based on their smile. While the week was really intense, we had a lot of fun working on this project.
The idea about this project was about helping children to find a book by topic they would like, so they could write an essay about it. But how do you know if a kid likes a topic? "Ah, how about a smile? If I like something I see, I usually start smiling!" This could be a perfect way to help children find a topic that fits their interest.
And so, the concept was born: choosing topics based on a smile and present a list of books that match with that topic. We would present images to the children about certain topics such as animals, sports, history etc. So, when a user smiles while a certain image is in front of them, they choose that topic (and maybe even a sub-topic after that), which eventually presents them a list of books based on their choices.
We discovered an opensource face recognition package that could be installed on a local server (no internet required). So we build ourselves a node.js server and set up the package to work with the our laptops camera. We aligned the Books-API from OBA (openbare bibliotheek van Amsterdam) and predefined a few topics in our prototype.
- node.js
- face recognition package
- Book-API from OBA
- Modern browsers like: Google Chrome or Firefox
- Java-script turned on
We intended to design this application to be run on a module in the public space of the library, so people could freely interact with it. Therefore, privacy matters a lot. The visitors of the library didn't choose to be continuously recorded while searching for books on the computer. With this package, no videos are stored or analysed at all for other purposes than finding books. Because the face recognition application runs completely local, all the data is safe. There are no video's recorded and the only requests from our server are sent to the books-API from OBA, which on their turn serve a list of relevant books.
- choosing topics with your smile
- presenting a list of books based on your choice
- showing results fitting to the users age (kids get kid books, adults get adult books)
- showing result based on emotion (more a gimmick then a feature)
- adjusting styling based on user age (children get a more playful design: different colors)
- people with motor impairments are also able to find books without using their hands.
- COVID-19 restraints, force us to design hands-free ways of interactions
However we only build this project in one week, it was more an experimental project, we realized the possibilities were endless. It really opened our eyes about new interaction methods. We both had never seen an application like this where a user was able to interact with a computer based on their face-expressions. We were both really curious what these technology's can bring us in te near future. Imagine interacting with your house, car or other devices with your face. This giving a whole new meaning to the definition inter-face :) (pun intended).
Curious about the face recognition software we used? Follow this link to their documentation