Adaptive Computation Time algorithm in Tensorflow
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.


Adaptive Computation Time algorithm in Tensorflow

This repo contains a file impementing ACT and inheriting from the abstract RNN class in the Tensorflow doccumentation here. It implements the Adaptive Computation Time Algorithm, described in this paper.

Also included in the repo is a model which uses this cell for language modelling on the Penn Tree Bank, a common dataset for evaluating language models. To run the file, you will need to download the dataset, which can be found here. The files you need are in the data/ directory.

In order to run the code in this repo you will need to have downloaded one of the daily Tensorflow binaries which you can find on their homepage.

To run:

run with the following parameters:

* model_size, Default: "small", Size of model to train, either small, medium or large(from the file)
* data_path, Default: None, full path to the data directory containing the ptb files
* model_path, Default: None, the full path of a pickled dictionary of weights(such as from a previously trained model)
 saved using to reload
* weights_dir, Default:None, full directory path to save weights into per epoch

Note - you will probably need some fairly heavy machinery to run the large config. The small model will comfortably run with around 4G of memory, although it will be pretty slow.