TensorLayer is a Deep Learning (DL) and Reinforcement Learning (RL) library extended from Google TensorFlow. It provides popular DL and RL modules that can be easily customized and assembled for tackling real-world machine learning problems.
TensorLayer grow out from a need to combine the power of TensorFlow with the right building modules for deep neural networks. According to our years of research and practical experiences of tackling real-world machine learning problems, we come up with three design goals for TensorLayer:
- Simplicity: We make TensorLayer easy to work with by providing mass tutorials that can be deployed and run through in minutes. A TensorFlow user may find it easier to bootstrap with the simple, high-level APIs provided by TensorLayer, and then deep dive into their implementation details if need.
- Flexibility: Developing an effective DL algorithm for a specific domain typically requires careful tunings from many aspects. Without the loss of simplicity, TensorLayer allows users to customize their modules by manipulating the native APIs of TensorFlow (e.g., training parameters, iteration control and tensor components).
- Performance: TensorLayer aims to provide zero-cost abstraction for TensorFlow. With its first-class support for TensorFlow, it can easily run on either heterogeneous platforms or multiple computation nodes without compromise in performance.
A frequent question regarding TensorLayer is that why do we develop this library instead of leveraging existing ones like Keras and Tflearn. TensorLayer differentiates with those with its pursuits for flexibility and performance. A DL user may find it comfortable to bootstrap with Keras and Tflearn. These libraries provide high-level abstractions that completely mask the underlying engine from users. Though good for using, it becomes hard to tune and modify from the bottom, which is necessary when addressing domain-specific problems (i.e., one model does not fit all).
In contrast, TensorLayer advocates a flexible programming paradigm where third-party neural network libraries can be used interchangeably with the native TensorFlow functions. This allows users to enjoy the ease of using powerful pre-built modules without sacrificing the control of the underlying training process. TensorLayer's noninvasive nature also makes it easy to consolidate with other wrapper libraries such as TF-Slim. However, flexibility does not come with the loss of performance and heterogeneity support. TensorLayer allows seamless distributed and heterogeneous deployment with its first-class support for the TensorFlow roadmap.
TensorLayer has install prerequisites including TensorFlow, numpy and matplotlib. For GPU support, CUDA and cuDNN are required. Please check here for detailed instructions.
If you already had the pre-requisites ready (numpy, scipy, scikit-image, matplotlib and nltk(optional)), the simplest way to install TensorLayer in your python program is:
[for stable version] pip install tensorlayer [for master version] pip install git+https://github.com/zsdonghao/tensorlayer.git (recommended)
The documentation [Online] [PDF] [Epub] [HTML] describes the usages of TensorLayer APIs. It is also a self-contained document that walks through different types of deep neural networks, reinforcement learning and their applications in Natural Language Processing (NLP) problems.
Chinese documentation is also available.
Your First Program
The first program trains a multi-layer perception network to solve the MNIST problem. We use the well-known scikit-style functions such as
test(). The program is self-explained.
import tensorflow as tf import tensorlayer as tl sess = tf.InteractiveSession() # Prepare data X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1,784)) # Define placeholder x = tf.placeholder(tf.float32, shape=[None, 784], name='x') y_ = tf.placeholder(tf.int64, shape=[None, ], name='y_') # Define the neural network structure network = tl.layers.InputLayer(x, name='input_layer') network = tl.layers.DropoutLayer(network, keep=0.8, name='drop1') network = tl.layers.DenseLayer(network, n_units=800, act = tf.nn.relu, name='relu1') network = tl.layers.DropoutLayer(network, keep=0.5, name='drop2') network = tl.layers.DenseLayer(network, n_units=800, act = tf.nn.relu, name='relu2') network = tl.layers.DropoutLayer(network, keep=0.5, name='drop3') # The softmax is implemented internally in tl.cost.cross_entropy(y, y_) to # speed up computation, so we use identity here. # see tf.nn.sparse_softmax_cross_entropy_with_logits() network = tl.layers.DenseLayer(network, n_units=10, act = tf.identity, name='output_layer') # Define cost function and metric. y = network.outputs cost = tl.cost.cross_entropy(y, y_) correct_prediction = tf.equal(tf.argmax(y, 1), y_) acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) y_op = tf.argmax(tf.nn.softmax(y), 1) # Define the optimizer train_params = network.all_params train_op = tf.train.AdamOptimizer(learning_rate=0.0001, beta1=0.9, beta2=0.999, epsilon=1e-08, use_locking=False).minimize(cost, var_list=train_params) # Initialize all variables in the session tl.layers.initialize_global_variables(sess) # Print network information network.print_params() network.print_layers() # Train the network, we recommend to use tl.iterate.minibatches() tl.utils.fit(sess, network, train_op, cost, X_train, y_train, x, y_, acc=acc, batch_size=500, n_epoch=500, print_freq=5, X_val=X_val, y_val=y_val, eval_train=False) # Evaluation tl.utils.test(sess, network, acc, X_test, y_test, x, y_, batch_size=None, cost=cost) # Save the network to .npz file tl.files.save_npz(network.all_params , name='model.npz') sess.close()
We provide many helper functions (like
test()) that is similar to Keras to facilitate your development; however, if you want to obtain a fine-grain control over the model or its training process, you can use TensorFlow’s methods like
sess.run() in your program directly (
tutorial_mnist.py provides more details about this). Many more DL and RL examples can be found here.
TensorLayer is a major ongoing research project in Data Science Institute, Imperial College London. TensorLayer contributors are from Imperial College, Tsinghua University, Carnegie Mellon University, Google, Microsoft, Bloomberg and etc. The goal of the project is to develop a compositional language while complex learning systems can be build through composition of neural network modules. The whole development is now participated by numerous contributors here.
🇬🇧If you are in London, we can discuss in person. Drop us an email to organize a meetup: firstname.lastname@example.org. 🇨🇳我们有官方的 中文文档。另外, 我们建立了多种交流渠道，如QQ 群和微信群（申请入群时请star该项目，并告知github用户名）. 需加入微信群，请将个人介绍和微信号发送到 email@example.com. 🇹🇭เราขอเรียนเชิญนักพัฒนาคนไทยทุกคนที่สนใจจะเข้าร่วมทีมพัฒนา TensorLayer ติดต่อสอบถามรายละเอียดเพิ่มเติมได้ที่ firstname.lastname@example.org.
- TensorLayer provides two set of Convolutional layer APIs, see (Professional)and (Simplified) on readthedocs website.
- If you get into trouble, you can start a discussion on Gitter, Help Wanted Issues, QQ group and Wechat group.
- Multi-layer perceptron (MNIST). A multi-layer perceptron implementation for MNIST classification task, see
- Denoising Autoencoder (MNIST). A multi-layer perceptron implementation for MNIST classification task, see
- Stacked Denoising Autoencoder and Fine-Tuning (MNIST). A multi-layer perceptron implementation for MNIST classification task, see
- Convolutional Network (MNIST). A Convolutional neural network implementation for classifying MNIST dataset, see
- Convolutional Network (CIFAR-10). A Convolutional neural network implementation for classifying CIFAR-10 dataset, see
- VGG 16 (ImageNet). A Convolutional neural network implementation for classifying ImageNet dataset, see
- VGG 19 (ImageNet). A Convolutional neural network implementation for classifying ImageNet dataset, see
- InceptionV3 (ImageNet). A Convolutional neural network implementation for classifying ImageNet dataset, see
- Wide ResNet (CIFAR) by ritchieng.
- More CNN implementations of TF-Slim can be connected to TensorLayer via SlimNetsLayer.
Natural Language Processing
- Recurrent Neural Network (LSTM). Apply multiple LSTM to PTB dataset for language modeling, see
- Word Embedding - Word2vec. Train a word embedding matrix, see
- Restore Embedding matrix. Restore a pre-train embedding matrix, see
- Text Generation. Generates new text scripts, using LSTM network, see
- Machine Translation (WMT). Translate English to French. Apply Attention mechanism and Seq2seq to WMT English-to-French translation data, see
- Deep Reinforcement Learning - Pong Game. Teach a machine to play Pong games, see
- Image Captioning - Reimplementation of Google's im2txt by zsdonghao.
- DCGAN - Generating images by Deep Convolutional Generative Adversarial Networks by zsdonghao.
- Merge TF-Slim into TensorLayer.
- Data augmentation with TFRecord. Effective way to load and pre-process data, see
- Data augmentation with TensorLayer, see
TensorLayer is releazed under the Apache 2.0 license.