Pull Request #113 breaks Issac Gym

[Tbarkin121]

I was modifying some of the rl_games code when I noticed newer version do not work with Issac Gym. Prior to this merge #113 things appear to be working correctly.

[Denys88]

Could you show your error please?
They idea of this pr is to move normalization to the model out of algorithm.
It worked for me, but it may not work with your changes and I am happy to help.

[Tbarkin121]

It doesn't have an error but it seems like the normalization being moved effects the output actions. When loading from a checkpoint and running the environment with test=True, the resulting actions seem scaled down by a large amount.

I had a suspicion that the normalization is missing from the models Isaac Gym is using, but I wasn't 100% sure where to start to fix this issue.

[ViktorM]

Hi @Tbarkin121! Can you please describe the repro steps - I've been training and testing a lot of different environments recently with the latest rl-games and they all worked fine for me.

[Denys88]

@Tbarkin121 what if you train in master and try to test in master?
I broke backward compatibility for old checkpoints now.

[Denys88]

@Tbarkin121 I've found the issue.
I removed load_path from the yaml config but Isaacgym never sent this parameter to me.
The fastest way is to fix it in the train.py in isaacgym:
runner.run({
'train': not cfg.test,
'play': cfg.test,
'checkpoint' : cfg.checkpoint # just add this line
})

[Denys88]

I added a few more fixes for the backward compatibility.