From 05e37f23035ee2afc2d1566e97a11094ce42844a Mon Sep 17 00:00:00 2001 From: Vinh Nguyen Date: Wed, 30 Oct 2019 12:13:41 +1100 Subject: [PATCH] adding documentation for automatic mixed precision training --- references/classification/README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/references/classification/README.md b/references/classification/README.md index f4ec967e8cb..8b37b8ff2a4 100644 --- a/references/classification/README.md +++ b/references/classification/README.md @@ -29,6 +29,16 @@ python -m torch.distributed.launch --nproc_per_node=8 --use_env train.py\ --lr-step-size 1 --lr-gamma 0.98 ``` +## Mixed precision training +Automatic Mixed Precision (AMP) training on GPU for Pytorch can be enabled with the [NVIDIA Apex extension](https://github.com/NVIDIA/apex). + +Mixed precision training makes use of both FP32 and FP16 precisions where appropriate. FP16 operations can leverage the Tensor cores on NVIDIA GPUs (Volta, Turing or newer architectures) for improved throughput, generally without loss in model accuracy. Mixed precision training also often allows larger batch sizes. GPU automatic mixed precision training for Pytorch Vision can be enabled via the flag value `--apex=True`. + +``` +python -m torch.distributed.launch --nproc_per_node=8 --use_env train.py\ + --model resnext50_32x4d --epochs 100 --apex +``` + ## Quantized ### Parameters used for generating quantized models: