loss decreasing is very slow

[eustcPL]

I try to use a single lstm and a classifier to train a question-only model, but the loss decreasing is very slow and the val acc1 is under 30 even through 40 epochs

[eustcPL]

can you help me? @Cadene

[Cadene]

Is it the open ended accuracy ?

With the VQA 1.0 dataset the question model achieves 40% open ended accuracy.

With VQA 2.0 dataset, it achieves 44%.

It could be a problem of overfitting, underfitting, preprocessing, or bug.

Did you try to change the number of parameters in your LSTM and to plot the accuracy curves ?

[Cadene]

Accuracy != Open Ended Accuracy (which is calculated using the eval code)

[eustcPL]

It is open ended accuracy in validation under 30 when training. Is it normal?

[Cadene]

I did not try to train an embedding matrix + LSTM.
Send me a link to your repo here or code by mail ;)

[eustcPL]

It's so weird. When use Skip-Thoughts, I can get much better result. Could you tell me what wrong with embedding matrix + LSTM?
Thank you very much!

[Cadene]

I just saw in your mail that you are using a dropout of 0.5 for your LSTM.

The cudnn backend that pytorch is using doesn't include a Sequential Dropout. That is why I made a custom API for the GRU.

I don't know what to tell you besides: you should be using the pretrained skip-thoughts model as your language only model if you want a strong baseline

What do you want to achieve ?

[eustcPL]

okay, thank you again! I will close this issue