Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


Train your own CTC model! This code was released with the lecture from the Bay Area DL School. PDF slides are available here.

Table of Contents

  1. Dependencies
  2. Data
  3. Running an example


You will need the following packages installed before you can train a model using this code. You may have to change PYTHONPATH to include the directories of your new packages.

The underlying deep learning Python library. We suggest downloading version 0.8.2 from

$tar xf <downloaded_tar_file>
$cd theano-*
$python install --user


pip install 'theano==0.8.2'

This is a wrapper over Theano that provides nice functions for building networks. Download version 1.1.2 from
Make sure you install it with support for hdf5 - we make use of that to save models.

$tar xf <downloaded_tar_file>
$cd keras-*
$python install --user


pip install 'keras==1.1.2'

Update the keras.json to use Theano backend:

vim ~/.keras/keras.json

Update the backend property

"backend": "theano"


$pip install lasagne <--user>

scipy Scipy needs to be version 0.18.1

pip install 'scipy==0.18.1'

This contains the main implementation of the CTC cost function.
git clone
To install it, follow the instructions on

This is a theano wrapper over warp-ctc.
git clone
Follow the instructions on for installation.

You may require some additional packages. Install Python requirements through pip as:
pip install soundfile
On Ubuntu, avconv (used here for audio format conversions) requires libav-tools.
sudo apt-get install libav-tools


We will make use of the LibriSpeech ASR corpus to train our models. While you can start off by using the 'clean' LibriSpeech datasets, you can use the script to download the entire corpus (~65GB). Use to convert any flac files to wav.
We make use of a JSON file that aggregates all data for training, validation and testing. Once you have a corpus, create a description file that is a json-line file in the following format:

{"duration": 15.685, "text": "spoken text label", "key": "/home/username/LibriSpeech/train-clean-360/5672/88367/5672-88367-0031.wav"}
{"duration": 14.32, "text": "ground truth text", "key": "/home/username/LibriSpeech/train-other-500/8678/280914/8678-280914-0009.wav"}

You can create such a file using

$python /path/to/LibriSpeech/train-clean-100 train_corpus.json
$python /path/to/LibriSpeech/dev-clean validation_corpus.json
$python /path/to/LibriSpeech/test-clean test_corpus.json

You can query the duration of a file using: soxi -D filename.

Running an example

Finally, let's train a model!

$python train_corpus.json validation_corpus.json /path/to/model

This will checkpoint a model every few iterations into the directory you specify. You can monitor how your model is doing, using

$python -d /path/to/model1 /path/to/model2 -s plot.png

This will save a plot comparing two models' training and validation performance over iterations. This helps you gauge hyperparameter settings and their effects. Eg: You can change learning rate passed to compile_train_fn in, and see how that affects training curves. Note that the model and costs are checkpointed only once in 500 iterations or once every epoch, so it may take a while before you can see updates plots.

Once you've trained your model for a sufficient number of iterations, you can test its performance on a different dataset:

$python test_corpus.json train_corpus.json /path/to/model

This will output the average loss over the test set, and the predictions compared to their ground truth. We make use of the training corpus here, to compute feature means and variance.

You can also visualize your model's outputs for an audio clip using:

$python audio_clip.wav train_corpus.json /path/to/model

This outputs: softmax.png and softmax.npy. These will tell you how confident your model is about the ground truth, across all the timesteps.


No description, website, or topics provided.







No releases published


No packages published