[Feature Request] Updating evogym Dependencies

[Okabe-Junya]

First of all, thank you for the wonderful software!! In our lab, we have been using evogym for our research.

Feature Request

I would like to update the dependencies without altering the functionality of evogym as much as possible.

Background

evogym is a fantastic software, however, its dependencies have not been updated. Because of this, we often struggle with installation and running experiments

• The Python version is outdated
• The README states Python 3.7/3.8, and setup.cfg states Python 3.6 or higher, but many of these versions are past their End Of Life
  • Note that support for Python 3.8 will end in October 2024
    • FYI: https://devguide.python.org/versions/
• Using legacy build systems like setup.py and setup.cfg
• Dependencies have not been updated.

Steps

To work on this issue, I think it should be divided into the following three major steps:

1. Introduce pyproject.toml
2. Update Python version
3. Migrate some libraries (for example, updating from numpy 1.24 to 1.25 or migrating from OpenAI Gym to Gymnasium may require significant effort)

Notes

If we proceed with addressing this issue, we might need to prepare a branch like v2.

[JohnStewardson]

Updating the environments to gymnasium would also be very helpful!

[Okabe-Junya]

Thanks @jagdeepsb for working on! Looking forward to update :)

[jagdeepsb]

Ok -- I made many of the requested changes, and evogym is now on pypi! Please test it and let me know if you run into any issues 😄 @Okabe-Junya @JohnStewardson

[JohnStewardson]

Trying to reproduce the results (to see if the optimal found robots are able to complete their task), I ran into problems. While the Walker-v0 worked fine other tasks were not completed by the specifically designed robots (for example for climbing, traversing). Trying to inspect the clipping/scaling, I modified the environments to print for debugging purposes. I noticed that changes made in the environments don't take effect. It seems as though the registration is not working as expected. After creating a copy of Walker-v0, naming it Walker-v1 and explicitly running code to register it, threw an error when running the run_ppo:
in the walk.py:
class WalkingFlat2(BenchmarkBase): ...
(same as WalkingFlat)
explicitly registering with this code:

from gymnasium.envs.registration import register

print("Registering environments")

register(
    id='Walker-v1',
    entry_point='evogym.envs.walk:WalkingFlat2',
    max_episode_steps=500
)

and then running run_ppo.py with Walker-v1 throws this error:

Traceback (most recent call last):
  File "run_ppo.py", line 52, in <module>
    best_reward = run_ppo(
  File "/home/jjfstlin/evogym/examples/ppo/run.py", line 25, in run_ppo
    vec_env = make_vec_env(env_name, n_envs=1, seed=seed, env_kwargs={
  File "/home/jjfstlin/anaconda3/envs/evogym/lib/python3.8/site-packages/stable_baselines3/common/env_util.py", line 125, in make_vec_env
    vec_env = vec_env_cls([make_env(i + start_index) for i in range(n_envs)], **vec_env_kwargs)
  File "/home/jjfstlin/anaconda3/envs/evogym/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 30, in __init__
    self.envs = [_patch_env(fn()) for fn in env_fns]
  File "/home/jjfstlin/anaconda3/envs/evogym/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 30, in <listcomp>
    self.envs = [_patch_env(fn()) for fn in env_fns]
  File "/home/jjfstlin/anaconda3/envs/evogym/lib/python3.8/site-packages/stable_baselines3/common/env_util.py", line 94, in _init
    env = gym.make(env_id, **kwargs)  # type: ignore[arg-type]
  File "/home/jjfstlin/anaconda3/envs/evogym/lib/python3.8/site-packages/gymnasium/envs/registration.py", line 741, in make
    env_spec = _find_spec(id)
  File "/home/jjfstlin/anaconda3/envs/evogym/lib/python3.8/site-packages/gymnasium/envs/registration.py", line 527, in _find_spec
    _check_version_exists(ns, name, version)
  File "/home/jjfstlin/anaconda3/envs/evogym/lib/python3.8/site-packages/gymnasium/envs/registration.py", line 424, in _check_version_exists
    raise error.VersionNotFound(message)
gymnasium.error.VersionNotFound: Environment version `v1` for environment `Walker` doesn't exist. It provides versioned environments: [ `v0` ].

@Okabe-Junya @jagdeepsb have you had similar problems?

[jagdeepsb]

Hey, it looks like the environment is not being registered by Gymnasium -- don't think this is an issue associated with Evogym. I recommend you follow the directory structure in our tutorial for registering a new environment (the tutorial hasn't yet been updated for gym --> gymnasium, but I think the directory/import structure is what's significant). Also, as done in the tutorial, you should inherit from EvoGymBase

[JohnStewardson]

I could solve the issue by uninstalling pip and rebuilding from source, now changes in the files take effect instead of being overwritten by the environment. Just in case someone else also has this problem.

[jagdeepsb]

Closing this as requested changes have been completed.