New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add samediff layer support (forward only, no output layers) #4675
Conversation
Something we will want to support: importing a layer definition (serialized SameDiff, TF, etc) into a DL4J layer. The current layer API may not easily support this in all cases. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks pretty good.
One thing I'l like to see if to minimize the number of functions you need to override.
One way we can do that is to define helper methods in BaseSameDiffLayer that you could use when creating the graph. These methods can save the parameter attributes and for calculating the values for weightKeys, biasKeys, paramShapes and initializeParams.
For example, we might be able to define an optional constructor (or overridable method) that takes a pair of arrays of SDVariables for weights and biases and use their sizes to calculate the shapes. The Chainer API takes that approach.
I'm not familiar with when the keys and shapes are required though, so it may not be possible to implement SameDiffLayer that way.
//No op constructor for Jackson | ||
} | ||
|
||
public abstract List<String> defineLayer(SameDiff sameDiff, SDVariable layerInput, Map<String,SDVariable> paramTable); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I felt this would be better defined as a List<SDVariable> instead of List<String>. That way you aren't just throwing away objects, and it's impossible to make a typo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's an idea oh how it could work (and a working unit test)
https://gist.github.com/wmeddie/809c602311503a2ebb2d2cd957daf09e
01a51a8
to
79af7a8
Compare
Starting from: #4426 - removes the backprop components and output/loss layers... I'll tackle those components separately.
Adds support for SameDiff layers: