[Feature Request] Add Pixel-level Observations for mujoco

[Benjamin-eecs]

Motivation

#107 (comment)

Solution

Init thought:

Add method render or off_screen_render in both mujoco_env.h

• dmc OffScreenRender
• gym OffScreenRender
• FrameStack [Feature Request] DMC framestack wrapper support #148 (comment)

Additional context

mujoco_py use MjRenderPool for multiprocess off-screen and on-screen render on multi-GPU

Road Map

• Go through their record.cc, get familar with off screen render with 3 different OpenGL context: GLFW, OSMesa and EGL
• Update BUILD file
• dm_control: Physics.render and Camera
• mujoco_py
• Add some unit tests (good to submit the first PR here);

Resource

dm_control

• Physics.render https://github.com/deepmind/dm_control/blob/main/dm_control/mujoco/engine.py
• Camera https://github.com/deepmind/dm_control/blob/main/dm_control/mujoco/engine.py

mujoco_py

• related issue Render off- and onscreen openai/mujoco-py#201
• mujoco_py.MjRenderContextOffscreen https://github.com/openai/mujoco-py/blob/master/mujoco_py/mjrendercontext.pyx
• off-screen rendering acceleration https://github.com/openai/mujoco-py/blob/master/mujoco_py/mjrenderpool.py

FrameStack Wrapper

• FrameStack

Checklist

• I have checked that there is no similar issue in the repo (required)

[zhongwen]

An internal project by https://github.com/Ericonaldo requires this feature. Can we bump the priority up? Thanks.