This is a repository for programming related assignments for job applicants or interns.
The code nn_backprop.py contains the code for the backpropagation algorithm to update the parameters of a shallow Neural Network. It is coded primarily using Numpy using sigmoid and rectified linear units as the activation functions. Training log of backpropagation algorithm:
Error: 0.25221945116214484
No. of Valid Points: 800
Train Accuracy at step 1: 56.99999999999999%.
Validation Accuracy at 1 step: 54.50000000000001%.
--------------------------------------------------
Error: 0.17190951666455048
No. of Valid Points: 800
Train Accuracy at step 25: 88.25%.
Validation Accuracy at 25 step: 86.5%.
--------------------------------------------------
Error: 0.12758249105712052
No. of Valid Points: 737
Train Accuracy at step 50: 88.75%.
Validation Accuracy at 50 step: 89.5%.
--------------------------------------------------
The predicted probability values is clipped at eps and 1.0-eps for numerical stability. When the predicted probability is outside the interval (eps, 1.0-eps), a mask will remove these points from updating the weights as the gradient at these points (after clipping) should be zero.
Please read the statistics assignment and the solution provided within. The script statistics_assignment_1.py has the solution in python code. In general, the solution may have numerical overflow as K becomes large so an asymptotic solution is also obtained.