Skip to content
/ glamor Public

Code for "Planning from Pixels using Inverse Dynamics Models"

Notifications You must be signed in to change notification settings

keirp/glamor

Repository files navigation

Goal-Conditioned Latent Action MOdels for RL (GLAMOR)

Setup

Create the Conda environment:

conda env create -f environment.yaml

Install additional dependencies:

pip install -e . dependencies/rlpyt git+git://github.com/mila-iqia/atari-representation-learning.git

Example

To train GLAMOR on Atari with default hyperparameters (same as used in the paper), use:

python main.py train_glamor_atari --use_wandb=False --run_path='runs/'

There is also an included notebook that can be used to train GLAMOR on a GridWorld task.

Structure

  • glamor
    • algos
      • batch_supervised.py (nn training loop)
      • batch_train_glamor.py (main algo logic)
    • datasets
      • frame_buffer.py (replay buffer that only stores each frame once in memory)
      • k_dist.py (code for sampling sequence lengths during training)
      • replay_buffer.py (uniform replay buffer)
    • envs (contains Atari, DM Control Suite, and GridWorld envs)
    • eval
      • label_compare_eval.py (evaluates policy in an env and returns statistics about achieved goals based on labels)
      • policy_video_eval.py (records videos of policies)
    • models
      • atari (pre-processing for Atari models)
      • basic (basic nn blocks)
      • encoder_lstm_model.py (main model class)
    • planner (contains the planning code)
    • policies (different policies like random, open and closed loop policies based on a plan, and eps-greedy)
    • samplers (code for sampling trajectories from the environment using a policy)
    • tasks (code for generating and sampling from task distributions)
    • train
      • scripts.py (main entry point, contains argument definitions)

Todo

  • Remove dependency on rlpyt and support normal gym environments.
  • Rewrite replay buffers to support non-visual goals.
  • Multi-processing for trajectory collection.

Bibtex

@article{paster2020planning,
title={Planning from Pixels using Inverse Dynamics Models}, 
author={Keiran Paster and Sheila A. McIlraith and Jimmy Ba},
year={2020},
eprint={2012.02419},
archivePrefix={arXiv},
primaryClass={cs.LG}
}

About

Code for "Planning from Pixels using Inverse Dynamics Models"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published