Added Neural Tanh Layer and corresponding unit test #3038
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | ||
| float64_t sum_j = 0; | ||
| for (int32_t j=0; j<num_inputs; j++) | ||
| sum_j += W(i,j)*W(i,j); |
There was a problem hiding this comment.
it's not 80s anymore. please use appropriate SIMD operations to do these things... or just use eigen.
|
this is a vanilla implementation. please consider using our linalg library and openmp at the least. |
|
@vigsterkr I was wondering why you guys didn't use the linalg library for the logistic layer so I just stuck with the same implementation. I'll modify this one to use it now. Thanks. |
| namespace shogun | ||
| { | ||
| /** @brief Neural layer with linear neurons, with a [tanh activation | ||
| * function](http://en.wikipedia.org/wiki/Hyperbolic_function). can be used as a |
There was a problem hiding this comment.
Can be used .... Capital C
Can you also put a reference to a paper here, where there is some evidence that this is useful? Thanks
|
Nice patch. |
|
news? |
|
Few things that block this being merged.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Tanh squasher is a very popular alternative to the sigmoid layer. It's widely used, although not as much as before thanks to relu's. It's definitely important for a complete neural nets package. :)