shouldn't it be D_real.backward(one)?

[ypxie]

  ```
   D_real.backward(mone)
   # train with fake
    noise = torch.randn(BATCH_SIZE, 128)
    if use_cuda:
        noise = noise.cuda()
    noisev = autograd.Variable(noise, volatile=True)  # totally freeze netG
    fake = autograd.Variable(netG(noisev).data)
    inputv = fake
    D_fake = netD(inputv)
    D_fake = D_fake.mean()
    D_fake.backward(one)

[caogang]

No, D_real is what we want to maximize, so we minimize the loss (-D_real)

[ypxie]

Thanks for your reply. That makes sense, but why does author of Wgan do the opposite in
https://github.com/martinarjovsky/WassersteinGAN/blob/master/main.py ?

[caogang]

Maybe, the output of net_d is the loss or error in the implementation of wgan. It is up to the definition of net_d

[ypxie]

Hello, Thanks for the explaining. I have a question, since you backward through the network twice, why is retain_variable=True not used in the code? And why not directly use D_cost.backward() once ?

[caogang]

I use retain_graph=True in autograd.grad to support twice backward which is the same as retain_variable=True. And I want get the gradient w.r.t. input without accumulating into .grad, just get the gradient. So I use autograd.grad instead of backward(). https://github.com/caogang/wgan-gp/blob/master/gan_toy.py#L204-L207

[eifuentes]

@ypxie @caogang see the WGAN author's comment in this issue martinarjovsky/WassersteinGAN#9 the two approaches are equivalent as long as you are consistent.