BUG: Scope of index variable in `for` loop

[oyamad]

Bug found by @tetrapod54:

In this block, the variable i in for i in range(n) can be referred from outside of the for loop:

    while i < max_iter and error > tol:

        for i in range(n):
            v_next[i] = np.max(mcm.state_action_values(i, v))

        error = np.max(np.abs(v_next - v))
        i += 1  # <- This is always equal to n

        v[:] = v_next

Try this:

for i in range(5):
    pass

print(i)

You will get 4.

The index variable for the for loop (or that for the while loop) has to be something other than i.

[mmcky]

thanks @oyamad and @tetrapod54

@shlff would you have time to review and setup a PR to fix this?