Final Loss equation

[hardianlawi]

Hi @yaringal !

I have read the paper and it is really amazing. Thanks for your team's hard-work!

However, I have a question regarding the final equation and also your keras implementation.

The equation above has 1/2 multiplied by the loss, but you didn't include it in the keras implementation.

I tried experimenting on it, and included 1/2 in the loss function, but it couldn't converge. I am wondering if the problem is in the paper or the keras implementation, because if I exclude the 1/2, it converges to the ground-truth std.

Best regards,

Hardian

[yaringal]

Please have a look at the updated NIPS version of the paper which has some typos corrected.
Yarin

[hardianlawi]

hi @yaringal thanks for your reply

Are you referring to this? I can't seem to find the one with the same title

[yaringal]

Ah apologies this is not updated online yet. It will be soon.
The equation above has a type - the implementation is correct.
Yarin

[divyansha]

According to the most recent version, the losses should still be multiplied by 1/2:

Why is a 1/2 missing from your code?

[divyansha]

I did the math from scratch and it does seem like there is agreement between code and the most recent version of paper.

Apologies for the trouble.

[John1231983]

Here you are https://arxiv.org/pdf/1703.04977.pdf

I am implementing it in pytorch. If you have done it, please share it

[John1231983]

@hardianlawi : I reproduced the result from keras using pytorch. YOu can look at https://colab.research.google.com/drive/1_zsmQguerz0iy0J9Uu2Cs7oEHhj0QoXH

However, the sigma2 does not work. Do you have any suggestion @yaringal ?

[Banyueqin]

@John1231983
you need to fix the code

[John1231983]

@Banyueqin : Thanks. But this is result after fix it

2627.982177734375
749.9542846679688
538.127685546875
401.61016845703125
308.06640625
241.04234313964844
191.45272827148438
153.8364715576172
124.77854919433594
102.02399444580078
84.0322265625
69.68601989746094
58.185302734375
48.94459533691406
41.494354248046875
35.47389221191406
30.60963249206543
26.680410385131836
23.508365631103516
20.950298309326172
tensor([8.7028]) tensor([5.3088]) 18.911481857299805

[Banyueqin]

@John1231983

result:

[John1231983]

No. We expect result is 10 and 0.

[lilaczhou]

@hardianlawi My reuslt goes to negative number after some steps,and I think it results from log_var(sigma),would you mind give me some suggestions?