Demos demonstrations of things that can go wrong when initializing weights
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Shows the "vanishing gradient" problem and "dying ReLU" problems, as well as the problems of the activations going to zero if we do the usual "small, random weight" initialization technique. Then we investigate the Xavier init, He init, and Batch normalization techniques for combatting this issue.