Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Add samediff layer support (forward only, no output layers) #4675
Starting from: #4426 - removes the backprop components and output/loss layers... I'll tackle those components separately.
Adds support for SameDiff layers:
left a comment
Looks pretty good.
One thing I'l like to see if to minimize the number of functions you need to override.
One way we can do that is to define helper methods in BaseSameDiffLayer that you could use when creating the graph. These methods can save the parameter attributes and for calculating the values for weightKeys, biasKeys, paramShapes and initializeParams.
For example, we might be able to define an optional constructor (or overridable method) that takes a pair of arrays of SDVariables for weights and biases and use their sizes to calculate the shapes. The Chainer API takes that approach.
I'm not familiar with when the keys and shapes are required though, so it may not be possible to implement SameDiffLayer that way.