- NVIDIA Tesla K80
- CUDA-8.0
- PyTorch-0.4.0
| Parameter | v1 | v2 | v3 |
|---|---|---|---|
| batch_size | 100 | 100 | 100 |
| epochs | 100 | 300 | 300 |
| eta | 1e-2 | 1e-2 | (1e-1, 1e-2, 1e-3) |
| normalize | Yes | Yes | Yes |
| train_time | ~18 mins | ~55 mins | ~ 55 mins |
| train_accuracy | 100% | 93% | 96.73% |
| test_error | 29.96% | 25.47% | 23.9% |
| Model | Architecture | Note |
|---|---|---|
| v1 |  |
|
| v2 |  |
Replace BatchNorm by Dropout and increase the number of epochs to 300 make the test_error decrease 4.49% compared to the v1 model |
| v3 |  |
Train the model with 3 different learning rates at different epochs make the test_error decrease 1.57% compared to the v2 model |