Skip to content

Clean, extensible implementation of MACAW [ICML 2021]

Notifications You must be signed in to change notification settings

eric-mitchell/macaw-min

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Minimal implementation of MACAW (ICML 2021)

This repo contains a simplified implementation of the MACAW algorithm (full code here) for easier inspection and extension.

Run the code with python impl.py.

Overview

This code trains MACAW on the simple Cheetah-Direction problem, which has only two tasks (forwards and backwards). impl.py contains example of loading the offline data (build_networks_and_buffers) and performing meta-training (loop in run.py). losses.py contains the MACAW loss functions for adaptation the value function and policy. utils.py contains the replay buffer implementation that loads the offline data.

Offline Data

The offline data can be downloaded from this link. It should be saved to a directory macaw_offline_data/cheetah_dir within the main project directory.

Citation

If our code or research was useful for your own work, you can cite us with the following attribution:

@InProceedings{mitchell2021offline,
    title = {Offline Meta-Reinforcement Learning with Advantage Weighting},
    author = {Mitchell, Eric and Rafailov, Rafael and Peng, Xue Bin and Levine, Sergey and Finn, Chelsea},
    booktitle = {Proceedings of the 38th International Conference on Machine Learning},
    year = {2021}
}

About

Clean, extensible implementation of MACAW [ICML 2021]

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages