What is grad_norms in AC_Network?

[yrlu]

Hi,

I come across your A3C implementation and find the following 2 lines in AC_network.py:

self.var_norms = tf.global_norm(local_vars)
grads,self.grad_norms = tf.clip_by_global_norm(self.gradients,40.0)

I wonder what's grad_norms for? It seems to me that it is not used.

Thanks!

[IbrahimSobh]

This is the norm value of gradients of the network, by monitoring this value, we know how much changes are applied to the network parameters during training

In case of gradient explosion, we usually use a simple trick, namely gradient clipping, to prevent too huge changes to the network parameters.

In the case above, 40.0 is used for clipping

Try to use different values (10, 20, 50, 80) and see the effect on the training and share the results if you like :)

[yrlu]

Thanks @IbrahimSobh , it helps!