MDP: mdp.py infinite loop for gamma = 1

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Run value_iteration for GridMDP with gamma = 1 (just like on Fig 17.3 of 
AI:MA second edition)
2. You'll get an infinite loop because of check @ line 101 after 58'th 
iteration because left-hand side is 0 and right-hand side is 0 too, so you get 
0 < 0 which is always false. 

What is the expected output? What do you see instead?
Expected output is stop on step 59

What version of the product are you using? On what operating system?
Python 2.7.1 on Mac OS X 10.7.2

Please provide any additional information below.
Most likely you should change < to <=

Also, there is no need to re-calculate this statement "epsilon * (1 - gamma) / 
gamma" each time in the loop, it does not depend on any loop variable.

Original issue reported on code.google.com by Stas.Kra...@gmail.com on 12 Nov 2011 at 1:00