A (growing) collection of Implementations in Tensorflow
Switch branches/tags
Clone or download

README.md

Tensorflow Libraries

TFLibrary is an open-source toolkit based on Tensorflow, with the design goals of modularity, flexibility and extensibility. It is in active development and used extensively in my own researches.

  • Data utility functions for tf.data, inspired from Google's NMT

  • Models simple wrapper class around models.

  • Modules a simplified version of sonnet.modules for modular model design.

    • Modules.attention attention modules takes in as input multiple inputs and compute attended features, currently includes matrix cross attention and BiDAF-style attention.
    • Modules.embedding embedding modules takes texts or text-ids and produce dense representations, currently includes matrix embedding, ELMO embedding, and GLOVE embedding.
    • Modules.encoders encoder modules encodes dense representations of input texts, currently includes LSTM encoder and BiDAF-style encoder.
    • Modules.transformer transformer modules are simplified / decomposed version of Transformer in Tensor2Tensor library, currently include Transformer encoder.
  • Tuner Hyper-parameter tuner using grid search as well as Bayesian Optimization, with support for multi-GPUs in parallel computations settings.

  • utils a collection of various utility functions.

    • utils.TrainingManager a light helper class for monitoring training progress, and providing signals for early stopping.
  • Bandit multi-armed bandit.

  • misc there are also a few other models, including DDPG, pointer network and its rnn_cell_impl.RNNCell wrapper version.

Installation

pip install -e .
pip install -r REQUIREMENTS.txt

Docker

# using `--no-cache` to avoid accidentally missing changes in parent tensorflow image
docker build -t tf-library:`git rev-parse --abbrev-ref HEAD` .

Versions

Version 0.9

  • Moved Tuner from utils/ to its own directory Tuner/, and changed both the interface and implementation of Tuner to support Bayesian Optimization based tuning and more flexible extension.