Add gradient output functionality in Neural Nets #2840
Conversation
| @@ -309,6 +309,18 @@ bool CNeuralNetwork::train_gradient_descent(SGMatrix<float64_t> inputs, | |||
|
|
|||
| float64_t e = compute_gradients(inputs_batch, targets_batch, gradients); | |||
|
|
|||
|
|
|||
| for (int32_t k=0; k<m_num_layers; k++) | |||
There was a problem hiding this comment.
@lisitsyn This for loop can be made a private member function if you want as it is getting repeated in lbfgs also.
|
Changes Unknown when pulling 9f55012 on sanuj:feature/grad_output into * on shogun-toolbox:develop*. |
| SGVector<float64_t> layer_gradients = network->get_section(gradients, i); | ||
| if (layer_gradients.vlen > 0) | ||
| { | ||
| SG_SINFO("Layer %i, Max Gradient: %g, Mean Gradient: %g.\n", i, |
There was a problem hiding this comment.
For convenience, can we output the layer name as well?
There was a problem hiding this comment.
Alright, i'll do it.
On 10 May 2015 23:10, "Sergey Lisitsyn" notifications@github.com wrote:
In src/shogun/neuralnets/NeuralNetwork.cpp
#2840 (comment):@@ -408,6 +420,19 @@ int CNeuralNetwork::lbfgs_progress(void* instance,
int n, int k, int ls)
{
SG_SINFO("Epoch %i: Error = %f\n",k, fx);
+
- CNeuralNetwork* network = static_cast<CNeuralNetwork*>(instance);
- SGVector<float64_t> gradients((float64_t*)g, network->get_num_parameters(), false);
- for (int32_t i=0; im_num_layers; i++)
- {
SGVector<float64_t> layer_gradients = network->get_section(gradients, i);if (layer_gradients.vlen > 0){SG_SINFO("Layer %i, Max Gradient: %g, Mean Gradient: %g.\n", i,For convenience, can we output the layer name as well?
—
Reply to this email directly or view it on GitHub
https://github.com/shogun-toolbox/shogun/pull/2840/files#r30004276.
sanuj
force-pushed
the
feature/grad_output
branch
from
9f55012
to
db043a7
Compare
|
@lisitsyn Done. Have a look. |
|
Looks good to merge. Thanks! |
Add gradient output functionality in Neural Nets
@lisitsyn Have a look. Currently I'm printing the max and mean gradients after each iteration. Something like: https://gist.github.com/sanuj/4994d1d8feccdf069400