Mode Collapse for toy dataset?

[hangg7]

Hi,

I was running your script in toy directory to reproduce results shown in paper, however ending up with mode collapse for both unimodal and multimodal data.

To deduce the reflection on initialization, I've run this test for 5 times on both datasets, results are pretty similar.

Any comments?

[LinkWoong]

Same

[swami1995]

Hi, I'm sorry I couldn't reply earlier.
Regarding the results. I'm not sure why you are getting such results. I just ran the code myself multiple times and my results seem pretty normal. Sometimes in the case of multiple gaussians the model does tend to oscillate between the results you show and the actual correct distribution but ultimately it does converge to the correct distribution once trained long enough. One thing you could try is learning rate decay if the problem persists in your case. But beyond that I'm not sure what's causing this issue in your case. But in the case of unimodal gaussian the behavior you show doesn't usually happen. I'm not sure what's causing it. I hope this was helpful.