A doubt at line 350 in dqn.cpp

[chiggum]

Firstly, sorry I posted this as an issue as I wasn't able to find your mail.

Doubt: I am not able to understand whether the solver takes a million steps at solver_->step(1) as mentioned in dqn_solver.txt or only a single step.
According to me 1 step == forward propagate, calculate errors, backpropagate errors and update filter weights.

Can you please clarify this?
Sorry I wasn't able to get this part of caffe documentation.

Thanks!
Regards.

[muupan]

No problem. Feel free to ask any question here.

solver_->Step(1) is a single step, an update from a minibatch of 32 transitions.

[chiggum]

Thanks a lot! I am also training an agent (rom: breakout) but facing quite a lot of difficulties as I am not using caffe but cudnn library. I'll then be asking you, my further queries here only. I hope to get some help.

Thanks a lot again.
Regards.

[chiggum]

Hi again,
As I said earlier, I am using breakout as the rom. I have written some code and it's working fine but unfortunately it's not learning as the slider in the game(the agent) either gets stuck to the left end of the screen or the right end.

After a lot of struggle, I am still unable to understand the reason for that. Do you have any clue why's that happening or can you have a look at my code please. I have added you as a collaborator in my private repo.

Thanks!
Regards.

[muupan]

I've not read your code yet, but the discussion of this thread can be of some help
https://groups.google.com/forum/#!topic/deep-q-learning/k1CzQxOi_v4

[chiggum]

Thanks a lot for that link! I think I can make it up from here.