Tensorflow implementation of "Language Modeling with Gated Convolutional Networks"
Switch branches/tags
Nothing to show
Clone or download
Permalink
Failed to load latest commit information.
assets image resized Jan 11, 2017
.gitignore basic flow Jan 1, 2017
README.md Summary changes for 0.12.1 Jan 16, 2017
conf_utils.py added readme Jan 11, 2017
data_utils.py added readme Jan 11, 2017
main.py Summary changes for 0.12.1 Jan 16, 2017
model.py Summary changes for 0.12.1 Jan 16, 2017

README.md

Language Modeling with Gated Convolutional Networks

This is a Tensorflow implementation of Facebook AI Research Lab's paper: Language Modeling with Gated Convolutional Networks. This paper applies a convolutional approach to language modelling with a novel Gated-CNN model.

Architecture

Architecture

Requirements

Usage

To train the model using the default hyperparameters:

$ python main.py
$ tensorboard --logdir=logs --host=0.0.0.0

Check main.py for tunable hyperparameter flags.

TODO

  • Replace NCE loss with Adaptive Softmax.
  • Remove restricted training on fixed sized sentences (20, for now) and extend to account for all varied sentence lenghts.
  • Implement Weight Normalisation for faster convergence.
  • Train extensively on deeper models to match the results with the paper.