Some kind of off-by-one mistake

[instance01]

Seen on gridworld (5x5 and 8x8). See following example on 8x8:

A good evaluation: 22222122222
A weird off-by-one evaluation: 222221222201010101010101010101010101010101010101010101010101010101010101010101010101010101 010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101 0101010101010101010101010101010101010101010101010101010101010101010101010101

So the last action before the goal goes left and right indefinitely.
I remember having the same issue in the Python implementation of BootlegAlphaZero.