Skip to content
No description, website, or topics provided.
Jupyter Notebook Python Dockerfile
Branch: master
Clone or download
Latest commit 487f7ab Jun 12, 2019

APEX PyTorch Cifar-10 Experiments


Batch size 128 | Wide Resnet | 10 epochs | Linear LR scheduler with Warmup

Google Colab (T4/K80)

Notebook snapshots stored in colab_snapshots subfolder.

Level GPU Time GPU Memory Validation Accuracy
O0 (Pure FP32) T4 57min 17s 7657 MB 86.75%
O1 (Mixed Precision) T4 31min 14s 4433 MB 85.25%
O2 (Mixed Precision) T4 29min 31s 4383 MB 88.90%
O3 (Pure FP16) T4 N/A 4347 MB Not Converged.
O0 (Pure FP32) K80 1h 43min 7s 6786 MB 88.44%

Kaggle (P100)

Kaggle Kernel used: APEX Experiment - Cifar 10.

Level Time GPU Memory Validation Accuracy
O0 (Pure FP32) 47min 01s 5677 MB 87.49%
O1 (Mixed Precision) 47min 34s 6283 MB 88.51%
O2 (Mixed Precision) 45min 34s 5665 MB 87.74%
You can’t perform that action at this time.