Have you run through dqn code？

[yikeqingli]

There are many errors in the code

[cyoon1729]

Yup, I am aware. I am currently refactoring this repository (in a new branch) and will fix them all.

[yikeqingli]

Thank you for your reply. There is one more question. What should I do when the buffer is full in D4PG? Only a piece of information is thrown out in your code. If I just use the remove function to remove one-half self.it_capacity every time the buffer is full, is it correct?
The code is like:
if idx> self.it_capacity:
             self.remove (0.5 * self.it_capacity)

[cyoon1729]

Hi @yikeqingli, sorry for the late reply. I have made some huge changes to the repository, addressing the following:

• Resolving memory leakage and bottlenecks; overall more correct and efficient distributed learning
• Reusability; I hoped to work toward more structural and general implementations of distributed RL, so incorporated abstract classes and reusable components. In particular, I will be moving onto implementing IMPALA and SEED.
• Using GPU for learner and CPU for workers actually works.

I don't have D4PG in my immediate To-do's, but it should be a fairly simple implementation from the reusable components. I would be happy to walk you through details, if you want!

[cyoon1729]

Closing this issue - feel free to reach out!