Fetching latest commit…
Cannot retrieve the latest commit at this time.

README.md

distributed_data_parallel.py and run.sh show an example using FP16_Optimizer with torch.nn.parallel.DistributedDataParallel and the Pytorch multiprocess launcher script, torch.distributed.launch. The usage of FP16_Optimizer with distributed does not need to change from ordinary single-process usage. Test via

bash run.sh