Python Shell
Switch branches/tags
Nothing to show
Clone or download
codekansas Merge pull request #42 from eshijia/master
Update the code to adapt to Keras2 API
Latest commit 14d6c31 Dec 17, 2017


Some code for doing language modeling with Keras, in particular for question-answering tasks. I wrote a very long blog post that explains how a lot of this works, which can be found here.

Stuff that might be of interest

  • Attentional LSTM, based on one of the papers referenced in the blog post and others. One application used it for image captioning. It is initialized with an attention vector which provides the attention component for the neural network.
  • Evaluation framework for the InsuranceQA dataset. To get this working, clone the data repository and set the INSURANCE_QA environment variable to the cloned repository. Changing config will adjust how the model is trained.
  • The LanguageModel class uses the config settings to generate a training model and a testing model. The model can be trained by passing a question vector, a ground truth answer vector, and a bad answer vector to fit. Then predict calculates the similarity between a question and answer. Override the build method with whatever language model you want to get a trainable model. Examples are provided at the bottom, including the EmbeddingModel, ConvolutionModel, and RecurrentModel.

Getting Started

# Install Keras (may also need dependencies)
git clone
cd keras
sudo python install

# Clone InsuranceQA dataset
git clone
export INSURANCE_QA=$(pwd)/insurance_qa_python

# Run
git clone
cd keras-language-modeling/

Alternatively, I wrote a script to get started on a Google Cloud Platform instance (Ubuntu 16.04) which can be run via

cd ~
git clone
cd keras-language-modeling

I've been working on making these models available out-of-the-box. You need to install the Git branch of Keras (and maybe make some modifications) in order to run some of these models; the Keras project can be found here.

The runnable program is This will create a models/ directory which will store a history of the model's weights as it is created. You need to set an environment variable to tell it where the INSURANCE_QA dataset is.

Finally, my setup (which I think is pretty common) is to have an SSD with my operating system, and an HDD with larger data files. So I would recommend creating a models/ symlink from the project directory to somewhere in your HDD, if you have a similar setup.

Serving to a port

I added a command line argument that uses Flask to serve to a port. Once you've installed Flask, you can run:

python serve

This is useful in combination with ngrok for monitoring training progress away from your desktop.


  • The official implementation can be found here