reinforce.rst

REINFORCE

In REINFORCE we don't rely on an external critic. Instead, we use the full Monte-Carlo return as our feedback signal.

The parametrized policy π_θ(a|s) is updated using the following policy gradients:

g(θ; S_t, A_t) = G_t ∇_θlog π_θ(A_t|S_t)

where G_t is the Monte-Carlo sampled return

G_t = R_t + γ R_t + 1 + γ²R_t + 2 + …

The sum is taken over all rewards up to the terminal state.

For more details, see section 13.3 of Sutton & Barto.

---

reinforce.py

reinforce.py