frames per state on each step

[fferreres]

Update - I just reviewed estimators.py and saw it uses the 4 frames already.

---

Denny, I was reading the paper, and not sure if they are feeding the past 4 frames as the "current" state in A3C. I thought they were following the protocols of previous papers, which might mean they feed the past 4 frames. I've also seen some other implementations using 4 frames (current + past 3) as current state.

My question is if your A3C you created allows or include past frames be passed as part of the current state. Karpathy also tried more creative things, like passing the difference of frames (which I liked, since you don't need to feed 2x or n times the features, but it nulls anything that didn't change).