Reinforcement Learning Policy Optimization: Deriving the Policy Gradient Update | Fan Pu Zeng

[giscus[bot]]

Reinforcement Learning Policy Optimization: Deriving the Policy Gradient Update | Fan Pu Zeng

Reinforcement learning algorithms that learn a policy (as opposed to implicit policy methods like (\epsilon)-greedy) optimize their policies by updating their policies in the direction of the gradient. However, the precise environment dynamics are not usually known to us, and the state space is usually also too large to enumerate, which means that we still cannot compute the gradient analytically. In this post, we derive the policy gradient update from scratch, and show how it can be approximated by sampling sufficiently many trajectories.

https://fanpu.io/blog/2022/deriving-the-policy-gradient-update/