My DDPG implementation as PL system

The Algorithm

Pseudocode

LunarLanderContinuous-v2 parameters:

Bad implementation

DDPG net:

BipedalWalker-v3 parameters:

Bad implementation

DDPG net:

Credits:

• DDPG | OpenAI
• Environments | OpenAI
• Optimization | Pytorch-Lightning
• Adam Grad - page 36 (Training NNs from Stanford's course)
• Kullback–Leibler divergence (YouTube video) - great
• Deep-Reinforcement-Learning-Hands-On-Second-Edition (page 512)
• 1 - Deep Deterministic Policy Gradient (DDPG): Theory and Implementation | Medium
• 2 - DDPG implementation | Medium
• Policy Gradient Algorithms | Lilian Weng's Blog
• DATASETS & DATALOADERS | PyTorch
• SAVING AND LOADING MODELS | PyTorch

Papers:

• @report{Silver2014, author = {David Silver and Nicolas Heess and Thomas Degris and Daan Wierstra and Martin Riedmiller}, keywords = {ICML,boring formatting information,machine learning}, title = {2014 - Silver - Deterministic Policy Gradient Algorithms.pdf}, year = {2014}, } (paper)

• @article{Sutton1999, author = {Richard S Sutton and David Mcallester and Satinder Singh and Yishay Mansour}, title = {Policy Gradient Methods for Reinforcement Learning with Function Approximation}, year = {1999}, } (paper)

• @article{Lillicrap2016, author = {Timothy P Lillicrap and Jonathan J Hunt and Alexander Pritzel and Nicolas Heess and Tom Erez and Yuval Tassa and David Silver and Daan Wierstra}, title = {CONTINUOUS CONTROL WITH DEEP REINFORCEMENT LEARNING}, url = { https://goo.gl/J4PIAz }, year = {2016}, } (paper)

• @article{weng2018PG, title = "Policy Gradient Algorithms", author = "Weng, Lilian", journal = "lilianweng.github.io/lil-log", year = "2018", url = "https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html" }