Dirichlet noise

[pathway]

From pp 24 of https://deepmind.com/documents/119/agz_unformatted_nature.pdf :

Additional exploration is achieved by adding Dirichlet noise 
to the prior probabilities in the root node s0, specifically 
P(s, a) = (1 − ε)p_a + εη_a, where η ∼ Dir(0.03) and ε = 0.25; 
this noise ensures that  all moves may be tried, but the
 search may still overrule bad moves

Searching this repo brings no matches for "dirichlet". I think you want numpy.random.dirichlet

Thanks for sharing this repo!

[pathway]

Btw how do you achieve exploration currently, is it purely via temperature?
Different exploration strategies are quite interesting.

[suragnair]

Yep, the exploration is purely via temperature in the code. Should be fairly easy to incorporate Dirichlet noise. It was a little unclear whether they use the noise in the evaluation stage.

[evg-tyurin]

@pathway there is sample implementation of Dirichlet noise in the following repo.

https://github.com/evg-tyurin/alpha-nagibator/blob/48b2ebd3ca272f388c13277297edbb60d98eb64b/MCTS.py#L191

The noise is used in self-play only.