Skip to content

JeanKaddour/PAML

Repository files navigation

Probabilistic Active Meta-Learning

In this work, we introduce task selection based on prior experience into a meta-learning algorithm by conceptualizing the learner and the active meta-learning setting using a probabilistic latent variable model.

Link to the paper

This repository implements the models and algorithms necessary to reproduce experiments (i)-(iii). To reproduce the results, you can run batch.sh (please do not change default values for certain parameters in run.py).

The core components of the repository are:

  • run.py: script to run the PAML algorithm including all parameters
  • env: directory for configuring and observing environments
    • controls.py: generates control signals
    • environment_configurator.py: configures dm_control environments
    • to.py: observes trajectories of the environments given controls
  • models: directory for the PAML model
    • meta_learner.py: trains the model and infers latent task variables
    • mlgp.py: the meta-learning (sparse variational) gaussian process model
    • tp.py: predicts trajectories (for evaluation)
  • utility_functions: directory for the in the paper used utility functions and baselines
    • paml.py: PAML
    • lhs.py: Latin Hypercube Sampling
    • uni.py: Uniform sampling
  • utils: directory for miscellaneous tools
    • algorithm_utils.py: separated key steps of the PAML algorithm
    • dataset.py: stores and prepares trajectory observations
    • evaluation.py: evaluates the model's performance on test tasks

Dependencies

This code was tested in Python 3.7. The dependencies can be found in requirements.txt.

Installation

  1. Download and install MuJoCo Pro 2.00
    • You need a license and you can request a trial license for 30 days
    • At installation time, dm_control, looks for the MuJoCo headers in ~/.mujoco/mujoco200_$PLATFORM/include
    • At runtime, dm_control looks for the MuJoCo license key file at ~/.mujoco/mjkey.txt
  2. Install all dependencies with pip install -r requirements.txt

Usage examples

# Under-specified cart-pole environment
python3 run.py --env_name="cartpole" --utility_function="PAML" --seed=1  --under_specified_system True --observed_config_space_dim=1
 
# Fully-specified cart-pole environment
python3 run.py --env_name="cartpole" --utility_function="PAML" --seed=1  

# Fully-specified pendubot environment
python3 run.py --env_name="pendubot" --utility_function="PAML" --seed=1  

# Fully-specified cart-double-pole environment
python3 run.py --env_name="cartdoublepole" --utility_function="PAML" --seed=1  

# Over-specified cart-pole environment
python3 run.py --env_name="cartpole" --utility_function="PAML" --seed=1  --over_specified_system True --observed_config_space_dim=3 --config_space_dim=2

Options for parameters

Parameters that require string values:

  • --env_name: 'cartpole', 'cartdoublepole', 'pendubot'
  • --utility_function: 'PAML', 'LHS', 'UNI'
  • --policy: 'ALTERNATE'
  • --initial_training_configurations : 'LHS', 'UNI'

Parameters that require boolean values:

  • --verbose: printing additional information
  • --evaluation: evaluation of the MLGP on a test task grid
  • --under_specified_system: enables an unobserved, stochastic configuration dimension
  • --oracle : initial training on the test task grid
  • --data_normalization : normalization of training data over all dimensions

Parameters intervals

The task paramater interval can be specified through the console, e.g.,

# By default, the following command runs an experiment with cart-pole tasks with pendulum mass in [0.5, 3.0] kg
python3 run.py --env_name="cartpole" --utility_function="PAML" --seed=1 --config_interval_lower_bound_dim_1=0.5 --config_interval_upper_bound_dim_1=3.0

In order to change the environment's parameterization (e.g., which configuration interval dimension corresponds to mass, length, radius, etc.), please have a look at env/environment_configurator.py

Cite

@inproceedings{kaddour2020paml,
  title={Probabilistic Active-Meta Learning},
  author={Kaddour, Jean and Saemundsson, Steindor and Deisenroth, Marc Peter},
  booktitle={Advances in Neural Information Processing Systems},
  year={2020}
}

About

Implementation of paper "Probabilistic Active Meta-Learning" (NeurIPS 2020).

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published