Sugar Tensor - A slim tensorflow wrapper that provides syntactic sugar for tensor variables
Sugar Tensor aims to help deep learning researchers/practitioners. It adds some syntactic sugar functions to tensorflow to avoid tedious repetitive tasks. Sugar Tensor was developed under the following principles:
- Don't mess up tensorflow. We provide no wrapping classes. Instead, we use a tensor itself so that developers can program freely as before with tensorflow.
- Don't mess up the python style. We believe python source codes should look pretty and simple. Practical deep learning codes are very different from those of complex GUI programs. Do we really need inheritance
and/or encapsulation in our deep learning code? Instead, we seek for simplicity and readability. For that,
we use pure python functions only and avoid class style conventions.
- tensorflow >= 0.12.0
- tqdm >= 4.8.4 ( for a console progressive bar )
pip install sugartensor
pip3 install sugartensor
docker installation : See docker README.md.
import sugartensor as tf # no need of 'import tensorflow'
All tensors--variables, operations, and constants--automatically have sugar functions which start with 'sg_' to avoid name space chaos. :-)
Chainable object syntax
Inspired by prettytensor library, we support chainable object syntax for all sugar functions. This should improve productivity and readability. Look at the following snippet.
logit = (tf.placeholder(tf.float32, shape=(BATCH_SIZE, DATA_SIZE)) .sg_dense(dim=400, act='relu', bn=True) .sg_dense(dim=200, act='relu', bn=True) .sg_dense(dim=10))
All returned objects are tensors.
In the above snippet, all values returned by sugar functions are pure tensorflow's tensor variables/constants. So, the following example is completely legal.
ph = tf.placeholder(tf.float32, shape=(BATCH_SIZE, DATA_SIZE) # <-- this is a tensor ph = ph.sg_dense(dim=400, act='relu', bn=True) # <-- this is a tensor ph = ph * 100 + 10 # <-- this is ok. ph = tf.reshape(ph, (-1, 20, 20, 1)).conv(dim=30) # <-- all tensorflow's function can be applied and chained.
Practical DRY (Don't repeat yourself) functions for deep learning researchers
We provide pre-defined powerful training and report functions for practical developers. The following code is a full mnist training module with saver, report and early stopping support.
# -*- coding: utf-8 -*- import sugartensor as tf # MNIST input tensor ( with QueueRunner ) data = tf.sg_data.Mnist() # inputs x = data.train.image y = data.train.label # create training graph logit = (x.sg_flatten() .sg_dense(dim=400, act='relu', bn=True) .sg_dense(dim=200, act='relu', bn=True) .sg_dense(dim=10)) # cross entropy loss with logit ( for training set ) loss = logit.sg_ce(target=y) # accuracy evaluation ( for validation set ) acc = (logit.sg_reuse(input=data.valid.image).sg_softmax() .sg_accuracy(target=data.valid.label, name='val')) # train tf.sg_train(loss=loss, eval_metric=[acc], ep_size=data.train.num_batch)
You can check all statistics through the tensorboard's web interface like the following.
If you want to write another more complex training module without repeating saver, report, or whatever, you can do that like the following.
# def alternate training func @tf.sg_train_func # <-- sugar annotator for training function wrapping def alt_train(sess, opt): l_disc = sess.run([loss_disc, train_disc]) # training discriminator l_gen = sess.run([loss_gen, train_gen]) # training generator return np.mean(l_disc) + np.mean(l_gen) # do training alt_train(log_interval=10, ep_size=data.train.num_batch, early_stop=False, save_dir='asset/train/gan')
Please see the example codes in the 'sugartensor/example/' directory.
You can add your own custom layer functions like the following code snippet.
# residual block @tf.sg_sugar_func def sg_res_block(tensor, opt): # default rate opt += tf.sg_opt(size=3, rate=1, causal=False) # input dimension in_dim = tensor.get_shape().as_list()[-1] # reduce dimension input_ = (tensor .sg_bypass(act='relu', bn=(not opt.causal), ln=opt.causal) .sg_conv1d(size=1, dim=in_dim/2, act='relu', bn=(not opt.causal), ln=opt.causal)) # 1xk conv dilated out = input_.sg_aconv1d(size=opt.size, rate=opt.rate, causal=opt.causal, act='relu', bn=(not opt.causal), ln=opt.causal) # dimension recover and residual connection out = out.sg_conv1d(size=1, dim=in_dim) + tensor return out # inject residual block tf.sg_inject_func(sg_res_block)
Namju Kim (firstname.lastname@example.org) at Jamonglabs Co., Ltd.