SYDE 522: Machine Intelligence course project on automated lip reading.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore
README.md
classify.py
feature_extraction.ipynb
pre_processing.py
vocab.py

README.md

Read My Lips

Lip reading is hard.

Directory structure

The pre_processing.py script is setup to handle a directory structure of:

data/align/<speaker>     # Align files for <speaker> go here.
data/videos/<speaker>    # Video files for <speaker> go here.
data/features/<speaker>  # Facial feature data for <speaker> is saved here. 

Saving facial feature data

First create the data/features/<speaker> directory for the speaker you're working with. Ensure you also have the align and video files for that speaker, as outlined above.

Now, all you need to so is run python pre_processing.py <speaker> to start generating and saving speaker data. You can stop at any time by pressing 'q' (it may take a few seconds to exit). You can pick up where you left off because the program is smart enough not to reprocess videos it already has data for.

Results

The paper describing our process and results can be found here.