Feature request: Gymnasium & PettingZoo support

[elliottower]

Hi, would it be possible to add PettingZoo support in the future? The current setup.py also uses OpenAI gym, which has not been maintained for a few years now, Gymnasium is the maintained version of it.

Gymnasium and PettingZoo are compatible with current RL training libraries (rllib, tianshou and CleanRL have already migrated, and stable-baselines3 will soon) as well as other tools such as Comet and WandB.

For information about upgrading and compatibility, see migration guide and gym compatibility. The main difference is the API has switched to returning truncated and terminated, rather than done, in order to give more information and mitigate edge case issues.

Adding PettingZoo support would be a bit more complicated, but if needed I would be happy to help look over code or answer any questions. It would be really helpful for future researchers, and we would be excited to list it in the third party environments lists (PettingZoo, Gymnasium).

[micahcarroll]

Hi @elliottower, does this look sufficient to you / what you had in mind?

[elliottower]

Hi @elliottower, does this look sufficient to you / what you had in mind?

Left some comments on that PR but just looked it over again and posting here as well, looks good to me.