Detecting lip popping noises to trigger an action.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
Makefile.inc
Makefile.linux
Makefile.mingw32
Makefile.osx
PopDetector.cpp
PopDetector.h
Readme.md
TemplateDetector.cpp
TemplateDetector.h
TssDetector.cpp
TssDetector.h
plugins.cpp
popTemplate.h
templates.h
vamp-plugin.list
vamp-plugin.map

Readme.md

Lip Popping Recognizer

This project implements a simple algorithm for recognizing when the user makes a popping noise with their lips. The plan is to use this in concert with my eye tracker as a way to click without using my hands.

It is currently being rewritten as a Vamp plugin in C++. I develop the Vamp plugin in Sublime Text, compile it to a dylib and then load it in Sonic Visualizer which is a great tool for developing audio recognition algorithms.

I plan on expanding this to recognize other mouth noises. This is an unsolved problem in audio recognition as it has very different goals from other domains like speech recognition:

  1. Noises are much easier to distinguish from each other. You can look at a spectrograph and identify if something is the noise you want, not so for a spoken word.
  2. Goal is to be resource efficient and real-time. Generally speech recognizers don't do this because good results require heavy processing.
  3. Highly reliable, very low false positives only a few false negatives. This is only possible because of the ease of distinguishing.

Sonic Visualizer Screenshot

Screenshot

The top row is a spectrograph, the middle is a debug visualization of the algorithm state, the bottom row is the waveform and some parameters. The red lines on the bottom row are when a lip pop can be recognized (if it was real time), the three lip pops and three non-pops are correctly classified.

This is the layout I use for debugging the algorithm, Sonic Visualizer allows me to inspect values and scroll and zoom around the test audio file.

Building

Run make -f Makefile.osx on osx, change the .osx to the correct Makefile if you are on another platform.