Theano implementation of CTC. Applied on TIMIT and a Toy dataset.
Python
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
LICENSE
README.md
ctc.py
edit_distance.py
ext_param_info.py
main_timit.py
main_toy_dataset.py
timit.py
toy_dataset.py

README.md

CTC-LSTM

This repositry contains an implementation of the CTC cost function (Graves et al., 2006).

  • CTC cost is implemented in pure Theano.

  • Supports mini-batch.

To avoid numerical underflow, two solutions are implemented:

  • Normalization of the alphas at each timestep
  • Calculations in the logarithmic domain

This repository also contains sample code for applying CTC to two datasets, a simple dummy dataset constituted of artificial data, and code to use the TIMIT dataset. The models are implemented using Blocks. Both datasets are implemented using Fuel.

The model on the TIMIT dataset is able to learn up to 50% phoneme accuracy using no handcrafted processing of the signal, but instead uses an end-to-end model composed of convolutions, LSTMs, and the CTC cost function.

Reference

Graves, Alex, et al. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. Proceedings of the 23rd international conference on Machine learning. ACM, 2006.

Credits

Alex Auvolat

Thomas Mesnard

Special thanks to

Mohammad Pezeshki