Multi gpu training

[constantinpape]

Multi-gpu training using DistributedDataParallel based on https://pytorch.org/tutorials/intermediate/ddp_tutorial.html.

This seems to be working (network is trained on example script), but I haven't tested if this implementation is doing exactly what it's supposed to.

[anwai98]

Hi @constantinpape,

Found the tutorial for the DistributedSampler implementation for the dataloaders: https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html