-
Notifications
You must be signed in to change notification settings - Fork 8.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Proposal] Allow multi-mode rendering for new Render API #3038
Comments
This seems extremely complex to implement with limited use cases. It could be possible to implement as a wrapper that reconstructed the original environment with a new render mode and on step both environments would be run to get both environment render results |
Let me clarify: |
That makes more sense from typing import Union, Tuple, Optional
import gym
from gym.core import ObsType, ActType
from gym.utils.env_checker import check_env
class MultiRenderEnv(gym.Env):
metadata = {
"render_modes": ["human", "rgb_array"]
}
def __init__(self, render_mode: Union[str, tuple[str]]):
self.action_space = gym.spaces.Box(0, 1)
self.observation_space = gym.spaces.Box(0, 1)
self.render_mode = render_mode
def reset(
self,
*,
seed: Optional[int] = None,
options: Optional[dict] = None,
) -> Tuple[ObsType, dict]:
super().reset(seed=seed)
self.observation_space.seed(seed)
return self.observation_space.sample(), {}
def step(
self, action: ActType
) -> Tuple[ObsType, float, bool, bool, dict]:
return self.observation_space.sample(), 0, False, False, {}
gym.register("multi-render-v0", entry_point=MultiRenderEnv)
env = gym.make("multi-render-v0", render_mode=("human", "rgb_array"))
# check_env(env.unwrapped) # this will fail |
Could you explain why you don't want to add this as a default feature? The passive checker( |
This seems like a niche feature that minimal users will need. What common cases do you imagine this being used? For changing how the render modes worked, we decided to move from |
@YouJiacheng here's the thing about the design philosophy of the core API - it mostly includes things that you generally want all envs to support. So whether you use Atari, Unity, Brax, MuJoCo, you know that you can do This feature would work for some environments, and not for others, right from the get-go. So it can't be reliably included in the core API. But here's the thing - if you're developing a custom environment, it actually makes little to no difference for you in the implementation. There's absolutely nothing stopping you from, for example, using |
Thanks for nice explanations from @pseudo-rnd-thoughts and @RedTachyon ! Using a custom(maybe private) API of custom environment is definitely better than making the general & public API complicated! I agree that we should keep the core API simple and minimum. I like this design philosophy - core API only provide minimum guarantee generally supported by all envs(with as less as possible violations), given the fact that the majority of RL algorithms only need this minimum guarantee. |
Actually #2989 is one of the use-cases. But it seems that use a wrapper is a better idea. |
Proposal
Optionally(i.e. for some environments) allow
Env.render_mode
to be atuple
to support rendering in multiple modes.Motivation
Multi-mode rendering might be useful, e.g. RGBD.
In old Render API, this is trivial, since we can simply call
env.render
with different modes multiple times.However, in new Render API, we must use multiple environment replicas with different render_mode.
Alternatively, user can add a new mode for each valid combination, which is not ergonomic.
Checklist
@younik
The text was updated successfully, but these errors were encountered: