Softsign activation function

[jeffin07]

Implemented Softsign activation and unit test for sotfsign

files changed

• activations.py
• test.py
  -../tests/tests.py

[ddbourgin]

Thanks, @jeffin07! I'm less inclined to keep adding activation functions, since I think we've already covered the majority of nonlinearities used in modern deep learning. If there's a compelling reason to add soft-sign (e.g., a paper / architecture that reports good results using soft-sign), let me know, but otherwise I think we're probably best keeping things as they are.

[jeffin07]

@ddbourgin i didn't see you closed #7 :) . I was thinking of doing a loss function do you have any suggestions ?

[ddbourgin]

That sounds great! Perhaps a cosine or KL-divergence loss? Depending on your enthusiasm, more sophisticated things like triplet loss or connectionist temporal classification loss would be awesome, though there's a reason why I've put them off ;-)

[jeffin07]

@ddbourgin thats great will close this PR now :)