Tools for capturing and analysing keyboard input paired with microphone capture
Switch branches/tags
Nothing to show
Clone or download
ggerganov [keytap2] Significant improvements
- Fix a bug in subbreak stuff
- Added option to set hints for different keys
- Added option to provide letter mask file

Code is a mess BUT I can solve the Keytap challenge with this !
Latest commit 3762781 Dec 8, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
build-em [view-full-gui] Remove cpp file from Emscripten compilation Dec 7, 2018
cmake Fix SDL2 not found Dec 4, 2018
data [keytap2] Adding sample n-gram.txt file Dec 8, 2018
imgui @ 0c4dee0 [keytap-gui] Emscripten support Nov 24, 2018
.gitmodules Adding CMake + ImGui Sep 4, 2018
.travis.yml Try Travis build with default compiler Dec 7, 2018
CMakeLists.txt Switch from c++17 to c++14 Dec 7, 2018
LICENSE Create LICENSE Nov 29, 2018
README.md Update README.md Dec 7, 2018
audio_logger.cpp [keytap-gui] Emscripten support Nov 24, 2018
audio_logger.h [keytap2-gui] Improving the keypress detection UI Oct 26, 2018
constants.h Adding 'subbreak.cpp' Oct 8, 2018
guess_qp.cpp Some refactoring to allow adjusting capture parameters Sep 16, 2018
guess_qp2.cpp Adding "guess_all" tool - can predict all keys Sep 16, 2018
imconfig-vtx32.h Display average waveforms of individual key strokes Sep 4, 2018
key_average_gui.cpp Start working on 'keytap2' Oct 8, 2018
key_detector.cpp Some minor stuff Sep 17, 2018
key_logger.cpp AudioLogger now stores captured frames Aug 27, 2018
key_logger.h AudioLogger now stores captured frames Aug 27, 2018
keytap-gui.cpp Fixing issues with GCC 5.4 and tuples Dec 7, 2018
keytap.cpp Fixing issues with GCC 5.4 and tuples Dec 7, 2018
keytap2-gui.cpp [keytap2] Significant improvements Dec 8, 2018
keytap2.cpp Fixing issues with GCC 5.4 and tuples Dec 7, 2018
play-full.cpp Add 'record-full' and 'play-full' tools Sep 30, 2018
play.cpp [play] Check if input file exists Sep 27, 2018
record-full.cpp Add 'record-full' and 'play-full' tools Sep 30, 2018
record.cpp Start working on 'keytap2' Oct 8, 2018
scale.cpp Some minor stuff Sep 17, 2018
subbreak.cpp [keytap2-gui] Initial subbreak integration Nov 24, 2018
subbreak.h [keytap2] Significant improvements Dec 8, 2018
view-full-gui.cpp Switch from c++17 to c++14 Dec 7, 2018
view-gui.cpp Switch from c++17 to c++14 Dec 7, 2018

README.md

kbd-audio Build Status

Description

This is a collection of command-line and GUI tools for capturing and analyzing audio data. The most interesting tool is called keytap - it can guess pressed keyboard keys only by analyzing the audio captured from the computer's microphone.

Check this blog post for more details:

Keytap: description and some random thoughts

Video: short demo of Keytap in action

CTF: can you guess the text being typed?

Build instructions

Dependencies:

  • SDL2 - used to capture audio and to open GUI windows libsdl
  • FFTW3 - some of the helper tools perform Fourier transformations fftw

Linux and Mac OS

git clone https://github.com/ggerganov/kbd-audio
cd kbd-audio
git submodule update --init
mkdir build && cd build
cmake ..
make

Windows

(todo, PRs welcome)

Tools

Short summary of the available tools. If the status of the tool is not stable, expect problems and non-optimal results.

Name Type Status
record text stable
record-full text stable
play text stable
play-full text stable
view-gui gui stable
view-full-gui gui stable
keytap text outdated
keytap-gui gui stable
keytap2 text development
keytap2-gui gui development
- extra -
guess_qp text experiment
guess_qp2 text experiment
key_detector text experiment
scale text experiment
subreak text experiment
key_average_gui gui experiment

Tool details

  • record-full

    Record audio to a raw binary file on disk

    Usage: ./record-full output.kbd


  • play-full

    Playback a recording captured via the record-full tool

    Usage: ./play-full input.kbd


  • record

    Record audio only while typing. Useful for collecting training data for keytap

    Usage: ./record output.kbd


  • play

    Playback a recording created via the record tool

    Usage: ./play input.kbd


  • keytap-gui

    Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool.

    Usage: ./keytap-gui input0.kbd [input1.kbd] [input2.kbd] ...

    Live demo (WebAssembly threads required)

    keytap-gui


  • keytap2-gui (work in progress)

    Detect pressed keys via microphone audio capture. Uses statistical information (n-gram frequencies) about the language. No training data is required. The 'recording.kbd' input file has to be generated via the record-full tool and contains the audio data that will be analyzed. The 'n-gram.txt' file has to contain n-gram probabilities for the corresponding language.

    Usage: ./keytap2-gui recording.kbd n-gram.txt

    keytap2-gui


  • view-full-gui

    Visualize waveforms recorded with the record-full tool. Can also playback the audio data.

    Usage: ./view-full-gui input.kbd

    view-full-gui


  • view-gui

    Visualize training data recorded with the record tool. Can also playback the audio data.

    Usage: ./view-gui input.kbd

    view-full-gui


Feedback

Any feedback about the performance of the tools is highly appreciated. Please drop a comment here.