Test various SGD algorithms on logistic regression and MLP, including
- vanilla SGD
- Nesterov Accelerated Gradient
We measure the performance of these SGD algorithms by comparing the training curve and validation error.
For more details about these algorithms, please refer to my blog (Chinese).