To make TensorLayer simple, we minimize the number of activation functions as much as we can. So we encourage you to use TensorFlow's function. TensorFlow provides tf.nn.relu
, tf.nn.relu6
, tf.nn.elu
, tf.nn.softplus
, tf.nn.softsign
and so on. More TensorFlow official activation functions can be found here. For parametric activation, please read the layer APIs.
The shortcut of tensorlayer.activation
is tensorlayer.act
.
Customizes activation function in TensorLayer is very easy. The following example implements an activation that multiplies its input by 2. For more complex activation, TensorFlow API will be required.
def double_activation(x):
return x * 2
tensorlayer.activation
identity ramp leaky_relu pixel_wise_softmax
identity
ramp
leaky_relu
pixel_wise_softmax
See tensorlayer.layers
.