This is the open source implementation of the representation learning and reinforcement learning method detailed in the following paper:
Marvin Zhang*, Sharad Vikram*, Laura Smith, Pieter Abbeel, Matthew J. Johnson, Sergey Levine.
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning.
International Conference on Machine Learning (ICML), 2019.
We use Pipenv for dependency management.
$ pipenv install
parasol/prior/blds.pyimplements our global dynamics prior
parasol/model/vae.pyis our model with modular priors including unit Gaussian and BLDS
parasol/control/lqrflm.pyimplements our control method
We provide a full example of the reacher experiment with the following files:
python experiments/vae/reacher-image/solar.pytrains our model and saves the weights
python experiments/solar/reacher-image/solar.pyloads the weights and runs SOLAR
Training the model is computationally expensive, slow, and
does not always produce consistent results.
Because of this, we also include a pretrained model in
These weights can be directly loaded and run with the second script. Please note that training a model
from scratch may lead to different policy performance, and multiple model training runs may be needed
in order to achieve good performance.