[Bug] Can no longer vectorize environments in the new release, but was working in v1.9.0

[mickelliu]

Search before asking

• I searched the issues and found no similar issues.

Ray Component

RLlib

What happened + What you expected to happen

I had a custom environment that relies heavily on vectorizing the environments on each worker. Before v1.10.0 it worked pretty nicely. Specifically, each remote worker sequentially runs my make_env function for num_envs_per_worker times. But now it seems that only one environment instance per worker is created, changing num_envs_per_worker has no effect. I have always set remote_worker_envs=False FYI.

Can anyone recall what's new in the new release that might have an impact on this?

Versions / Dependencies

Ray=1.10

Reproduction script

Unfortunately my project is currently closed-source, but I am willing to try anything you suggested.,

Anything else

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[mickelliu]

Actually, me and my colleague have figured it out... this is an issue related to convert_to_base_env in base_env.py and the fix was provided in #21670 but was not included in the latest release.

env.to_base_env() needs the argument num_envs in order to vectorized the environments correctly.

Is it possible that we can soon issue another release because I felt like this is a prominent issue that is going to affect a lot of people...