The reason for clipped reward in V-trace

[benlin1996]

For example, in the network.py in dmlab, you use clipped_reward (line 112). Can I know the reason why you construct the reward and the network this way? And is there any constraint on my reward setting in the environment.

Thanks

[lespeholt]

For this project we didn't try to innovate much on the networks. The network is really from the IMPALA work: https://arxiv.org/abs/1802.01561

For some games it may make sense to not clip or to represent the reward as a onehot. However, I haven't found it to be very important. It makes the most sense when the same object in a game can have different rewards depending on other factors in the game.

Clipping the reward for the loss is important though: https://github.com/google-research/seed_rl/blob/master/agents/vtrace/learner.py#L94

[benlin1996]

Thanks for your explanation