NaN output with distributed training

[choltz95]

Hi, thanks for this project. Just opening for future investigation. I am finding that training with more than one GPU using the basic pytorch DPP demo on CIFAR-10 results in NaN outputs after a few epochs. Training using a single gpu works great within the DPP framework.

The implementation from [meliketoy](https://github.com/meliketoy/wide-resnet.pytorch works fine, but uses more gpu memory.

[choltz95]

looks like that other repo also suffers from the same issue, just takes longer.

[choltz95]

Just in case someone ever has a similar issue with DPP: need to ensure proper usage of DPP.no_sync. In my case, it was necessary for adversarial training.