distributed pytorch example repository

pyTorch supports distributed training and some example codes are provided at the official website. This repository contains more specific and concrete example codes for various distributed training scenario. The list of codes are described bellow.

Multi-Node & Multi-GPU DP (/multinode-DP)

• example code for multi-node multi-gpu DP with spwan : multinodeDP.py

Multi-Node & Multi-GPU MP (/multinode-MP)

• example code for multi-node multi-gpu MP with spwan : multinodeMP.py