[Feature Request] Support for Gym(nasium) v0.22+

[chrisyeh96]

Is there any estimated timeline for when OpenAI Gym v0.22+ will be supported?

gym v0.22 was understandably a large breaking change, but it would be great to know when SB3 might start supporting it.

[araffin]

See PR #780

duplicate of #840

PS: if you want faster support, we need help to update our notebooks/tutorials

[araffin]

Gymnasium support (by default) is here: #1327
With support for gym 0.21/0.26 and gymnasium env.

[araffin]

I just released an alpha version of the Gymnasium branch on PyPi:

# sb3 contrib installs sb3 too
pip install "sb3_contrib>=2.0.0a1" --upgrade

Documentation is available here: https://stable-baselines3.readthedocs.io/en/feat-gymnasium-support/

[ReHoss]

Hi @araffin ,

First thank everyone for the update to Gymnasium!

Could you or someone provide a shortlist of fixes to apply to our Gym 0.21 custom environments in order to keep reproducibility in the context of sb3 ?

Thank you for your consideration,
Best,

[araffin]

Could you or someone provide a shortlist of fixes to apply to our Gym 0.21 custom environments in order to keep reproducibility in the context of sb3 ?

What do you mean exactly?
You want to keep using gym 0.21 or you want to upgrade to gymnasium and get the exact same results with SB3 as before when using the same setup (same machine, same os, same virtual env, same random seed)?

[ReHoss]

Could you or someone provide a shortlist of fixes to apply to our Gym 0.21 custom environments in order to keep reproducibility in the context of sb3 ?

What do you mean exactly? You want to keep using gym 0.21 or you want to upgrade to gymnasium and get the exact same results with SB3 as before when using the same setup (same machine, same os, same virtual env, same random seed)?

Yes I exactly want to upgrade to Gymnsasium while keeping same results when using same setup. I am just wondering if 0.21 custom envs adapt directly to 0.28 or do I need to fix some things?

[araffin]

I am just wondering if 0.21 custom envs adapt directly to 0.28 or do I need to fix some things?

It might work out of the box (and you will need #1486, we are missing reviewers for that PR) but I would say there are no warranty as Gymnasium changed a lot of things internally and might require a newer version of numpy.

As written in the documentation, we can only guaranty exact reproducibility for a given SB3/gym version.
However, performance should be the same (when comparing quantitative evaluations).

[ReHoss]

I am just wondering if 0.21 custom envs adapt directly to 0.28 or do I need to fix some things?

It might work out of the box (and you will need #1486, we are missing reviewers for that PR) but I would say there are no warranty as Gymnasium changed a lot of things internally and might require a newer version of numpy.

As written in the documentation, we can only guaranty exact reproducibility for a given SB3/gym version. However, performance should be the same (when comparing quantitative evaluations).

Thanks, I'll provide some feedback at some point.

EDIT:

• Here is a migration guide from gymnasium: https://gymnasium.farama.org/content/migration-guide/.