multi-GPU issues around --no_vgg_loss

[elfprince13]

This code:

pix2pixHD/models/pix2pixHD_model.py

Lines 160 to 172 in db4a24d

	loss_G_GAN_Feat = 0
	if not self.opt.no_ganFeat_loss:
	feat_weights = 4.0 / (self.opt.n_layers_D + 1)
	D_weights = 1.0 / self.opt.num_D
	for i in range(self.opt.num_D):
	for j in range(len(pred_fake[i])-1):
	loss_G_GAN_Feat += D_weights * feat_weights * \
	self.criterionFeat(pred_fake[i][j], pred_real[i][j].detach()) * self.opt.lambda_feat
	# VGG feature matching loss
	loss_G_VGG = 0
	if not self.opt.no_vgg_loss:
	loss_G_VGG = self.criterionVGG(fake_image, real_image) * self.opt.lambda_feat

Causes trouble when using --no_vgg_loss with multiple GPUs (and I suspect would also cause trouble for --no_ganFeat_loss), because the value 0 is not compatible with the scatter/gather APIs used by torch.nn.DataParallel. I suspect it needs to be Variable containing a 0-dimensional tensor, but I haven't quite figured out how to make it work.

[WayneCho]

I think loss_G_VGG = torch.zeros(1).cuda() may work.