Multi-GPU support and multi-threaded data loading

[rohanchabra]

I think it will be good to use nn.DataParallel
self.model = nn.DataParallel(self.model)

Maybe also add multi-threading in dataloader in your utils.py (This can decrease training time by 4 times atleast)

loader = DataLoader(dataset, batch_size= batch_size,
shuffle=True, num_workers= 4)

[voeykovroman]

Yeah! Thank you @rohanchabra! Indeed it would be a significant improvement. I'll try to add these features later next week after implementing the rest of our planned "To do" list!

[NikolasEnt]

Good idea, we'll add it.
Optimal number of threads in dataloader depends on hardware and use of multi-threading reduce predictability, so, num_workers should be set as an optional parameter.

[NikolasEnt]

a88d674