Value Iteration prerequisites: finite Markov decision processes the agent knows environment dynamics accurately $p(r, s' \mid s, a)$ for every variable Monte Carlo prerequisites: Markov decision processes episodic tasks