A question about the test_env_seed

[MagiFeeney]

I found in your code, no matter what changes made to the main seed, it won't influence the test_env_seed at all. In other words, with different training seeds, it only evaluate on a same seed (test_seed: int = 1841). Is there any intention for this choice? Or should I also split a more seed for evaluation? Something might like this:

# original seeding at https://github.com/facebookresearch/denoised_mdp/blob/main/main.py#L546C5-L546C104
torch_seed, np_seed, data_collect_env_seed, replay_buffer_seed = split_seed(cast(int, cfg.seed), 4)
# to
torch_seed, np_seed, data_collect_env_seed, replay_buffer_seed, test_env_seed = split_seed(cast(int, cfg.seed), 5)

[ssnl]

It was an intentional choice so that training configs don't affect test settings. You can split that if you want. Just be mindful of the behavior changes.

[MagiFeeney]

Thanks for yours explanation. However, my major concern is that only testing on a fixed seed would lead increase divergence between the training. To my knowledge, current RL algorithms either test on the same env used for training or at least keeps a constant shifting (i.e. test_env_seed = const + train_env_seed). I don't really have a clue how much those seedings affect the performance. Hope you can make some clarification.

[ssnl]

I'm not sure how test seeds can cause training divergence.

You can make an argument for either case and I think they are both valid.

[MagiFeeney]

Ok, I will experiment more see if there is an answer. Thanks for you time.