Update HER + update to gym 0.26+

[arjun-kg]

To be merged after next SB3 release. Notebooks run with SB3 at this fork (DLR-RM/stable-baselines3#780)

1. Updated to new seed method in one file
2. Updated HER to be compatible with latest SB3

[araffin]

Thanks again =)

[tlpss]

The wandb integration is not working for gym v0.26 yet, see wandb/wandb#4305. A PR seems to be in place so I'm hoping this gets merged and released soon.

[tlpss]

@araffin 3 notebooks are not working yet for gym v0.26:

• pybullet_envs -> no word on gymv0.26 yet Bullet3 in progress (compatibility with Gym 0.26.0 bulletphysics/bullet3#4335)
• wandb -> integration for gymv0.26 in progress ([CLI]: Integration with OpenAI gym is broken due to missing ImageEncoder in the most recent gym version wandb/wandb#4305)
• rl-zoo -> integration in progress (Add Gym 0.26 support DLR-RM/rl-baselines3-zoo#256)

[araffin]

pybullet_envs -> no word on gym v0.26 yet, repo seems to have low activity (pybullet is no longer actively developed after all)

Pull request is here: bulletphysics/bullet3#4335

Pybullet is no longer actively maintained but that doesn't mean it's not usable/useful (especially pybullet envs are more interestings and challenging than the mujoco equivalents).

[tlpss]

Pull request is here: bulletphysics/bullet3#4335

Ah my bad, I was looking in the pybullet-gym repo.

Pybullet is no longer actively maintained but that doesn't mean it's not usable/useful

Couldn't agree more! I have been using it extensively the past months.

[araffin]

pybullet_envs -> no word on gymv0.26 yet Bullet3 in progress

we could activate the compatibility in the meantime, no?

Nevermind, gym breaks pybullet by breaking changes in its registry...

[araffin]

pybullet_envs -> no word on gymv0.26 yet Bullet3 in progress (compatibility with Gym 0.26.0 bulletphysics/bullet3#4335)

We should probably patch it the way I did for the RL Zoo: bulletphysics/bullet3#4368 (comment)

EDIT: done