Consider using qtransform_completed_by_mix_value.

[fidlej]

Thanks for the nice project.
Have you tried using the default qtransform_completed_by_mix_value for the gumbel_muzero_policy?

The qtransform_by_min_max gives zero values to unvisited actions. That does not have a good theoretical justification.

[NTT123]

Hi @fidlej, thanks for informing me about this issue. I will take a deeper look into these transforms.

[NTT123]

The agent is much stronger when training with qtransform_completed_by_mix_value. Thank you again for your suggestion!