Activation threshold for nonlinearity

[run2]

Hi
Is it possible (does it make sense ?) to have an activation threshold (like axion hill) for tanh or sigmoid ? The get_output_for method in Conv layer will always return an output (i.e a neuron will fire unless it is Relu) - right ? (however small it is).

[dnouri]

Activation functions are defined in lasagne.nonlinearities. It should be easy enough to try this out and make a tanh version with a threshold. Feel free to reopen if you think this is a nolearn issue.