[Feature Request] Refactor mujoco_env.h with dm composer callback API

[Trinkle23897]

Motivation

#59 (comment)

We are going to refactor the MujocoEnv class with deepmind composer API. It is more general with various kinds of robotics research.

Resources

• https://arxiv.org/pdf/2006.12983.pdf page 16
• composer task: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/task.py
• composer environment: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/environment.py

[Benjamin-eecs]

Basically class task serve as hooks in MujocoEnv with following methods:

Class task

• initialize_episode_mjcf
• after_compile
• initialize_episode
• before_step
• before_substep
• after_substep
• after_step

DeepMind Control Suite, 14 envs

• Acrobot (@Benjamin-eecs, dmc acrobot #129)
• Ball in cup (@Benjamin-eecs, dmc ball_in_cup #124)
• Cart-pole (@Benjamin-eecs, dmc cartpole #137)
• Cheetah (@yufansong, dmc cheetah #110)
• Finger (@yufansong, dmc finger #118)
• Fish (@Benjamin-eecs, dmc fish #134)
• Hopper (@Trinkle23897, Add dm_control suite Hopper #107)
• Humanoid (@Benjamin-eecs, dmc humanoid #132)
• Manipulator (@yufansong, dmc manipulator #128)
• Pendulum (@yufansong, dmc pendulum #122)
• Point-mass (@yufansong, dmc point mass #123)
• Reacher (@yufansong, dmc reacher #115)
• Swimmer (@Benjamin-eecs, dmc swimmer #140)
• Walker (@yufansong, dmc walker #114)

Extra envs in dm_control.suite

• Dog
• Humanoid_CMU (@Benjamin-eecs, dmc humanoid_CMU #145)
• Lqr (@Benjamin-eecs )
• Quadruped
• Stacker

dm_control.locomotion

• Navigating a corridor with obstacles
• Maze navigation and foraging
• Multi-Agent Soccer

Misc

• Apply name2id (@Trinkle23897, Replace hardcode index with mj_name2id #125)
• Class variation
• Class observables
• dm_control.manipulation

Resources

• control.Environment: https://github.com/deepmind/dm_control/blob/main/dm_control/rl/control.py
• control.Task: https://github.com/deepmind/dm_control/blob/main/dm_control/suite/base.py
  https://github.com/deepmind/dm_control/blob/main/dm_control/rl/control.py
• composer.Environment: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/environment.py
• composer.Task: https://github.com/deepmind/dm_control/blob/main/dm_control/composer/task.py

[kevinzakka]

I'll offer feedback as soon as I'm free :) Thanks for initiating!