Robotic Control Using Model Based Meta Adaption

This is the official code for the paper "Robotic Control Using Model Based Meta Adaption" accepted to ICRA 2023. The code allows users to reproduce the results reported in the paper. Detailed results can be found in this report.

Installation

1. Create and activate new environment

conda create --name mac python=3.9

conda activate mac

2. navigate to this directory and install the required packages

cd mac

pip install -r requirements.txt

3. install mbmrl_torch as a package

cd mac

pip install -e .

4. OPTIONAL: If you are using a computer with Apple silicon (e.g. Apple M1) follow these instructions openai/mujoco-py#682 to install mujoco while using the following script:

sh install-mujoco.sh

How to test MAC?

Configuration

1. Go to mbmrl_torch/configurations/conf/famle/config_famle.yaml and define "wandb_entity" by inserting your username for https://wandb.ai
2. Go to mbmrl_torch/configurations/conf/fomaml/config_fomaml.yaml and define "wandb_entity" by inserting your username for https://wandb.ai
3. Optional: Adust config as received through hyperparameter search (e.g. in famle/stored_configs=test_mac_ant_cripple).

Execution

Ant Disabled

python3 mbmrl_torch/launcher/run.py +famle/stored_configs=test_mac_ant_cripple

Ant Gravity

python3 mbmrl_torch/launcher/run.py +famle/stored_configs=test_mac_ant_gravity

Halfcheetah Blocks

python3 mbmrl_torch/launcher/run.py +famle/stored_configs=test_mac_halfcheetah_blocks

Halfcheetah Disabled

python3 mbmrl_torch/launcher/run.py +famle/stored_configs=test_mac_halfcheetah_cripple

Reproducibility of Results

Please note that although the number of sources of nondeterministic behavior is limited, experiment results vary across different hardware configurations (see: https://pytorch.org/docs/stable/notes/randomness.html, https://discuss.pytorch.org/t/reproducibility-over-different-machines/63047). Therefore, to reproduce the papers results exactly, a hyperparameter search must be executed for each algorithm-env-configuration.

How to Make a Hyperparameter Search?

Ant Disabled

wandb sweep mbmrl_torch/configurations/sweep_test_mac_ant_cripple.yaml

Ant Gravity

wandb sweep mbmrl_torch/configurations/sweep_test_mac_ant_gravity.yaml

Halfcheetah Blocks

wandb sweep mbmrl_torch/configurations/sweep_test_mac_halfcheetah_blocks.yaml

Halfcheetah Disabled

wandb sweep mbmrl_torch/configurations/sweep_test_mac_halfcheetah_cripple.yaml

Video with task description

AntMac_HQ.mp4