How to train on multi GPU

[KaleidoZhouYN]

Hi:
I have trained with your code on single GPU and get a pretty nice result,excellent job!!!!

also I met something trouble when i want to accelerate with Multi GPU:

And the source code with multiGPU makes me confused:
options/base_options.py: if len(self.opt.gpu_ids) > 0:
options/base_options.py: torch.cuda.set_device(self.opt.gpu_ids[0])

Any advice?Thank you very much

[junyanz]

You can set the gpu_ids as 0,1,2. len(self.opt.gpu_ids) > 0
checks if there are more than one gpu ids.
torch.cuda.set_device(self.opt.gpu_ids[0]) sets the main device as the first gpu.
See here for more discussion. The multi-GPU code should work with instancenorm. We haven't implemented the synchronized batchnorm as mentioned here

[KaleidoZhouYN]

@junyanz Yes,I append the args --gpu_ids=0,1,2 and did not change the type of "norm".
will there be some trouble with pytorch version or something else?

[junyanz]

it should be fine. You can see which norm you are using, from the output log.