baselines

This example includes the implementations of the following reinforcement learning algorithms:

• ES
  • Cross Entropy Method (CEM)
  • Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
  • OpenAI-ES
• RL
  • Vanilla Policy Gradient (VPG)
  • Proximal Policy Optimization (PPO)
  • Deep Deterministic Policy Gradients (DDPG)
  • Twin Delayed DDPG (TD3)
  • Soft Actor-Critic (SAC)

Download loggings for reproducibility

Because the loggings and checkpoint files are very large, putting them in the repo directly leads to inconveniently slow cloning, so we host the logging files in Dropbox.

• Download the file
• Put all_logs.tar.gz under the directory lagom/baselines
• Run python unzip_logs.py, this will automatically exact all logs folders under each algorithm folder.

Benchmarks

ES

Model-free RL

FAQ:

• How to train with dm_control environments?
  • Modify experiment.py: use dm2gym wrapper, e.g.

  from gym.wrappers import FlattenDictWrapper
  from dm_control import suite
  from dm2gym import DMControlEnv
  
  config = Config(
      ...
      'env.id': Grid([('cheetah', 'run'), ('hopper', 'hop'), ('walker', 'run'), ('fish', 'upright')]),
      ...
      )
  
  def make_env(config, seed):
      domain_name, task_name = config['env.id']
      env = suite.load(domain_name, task_name, environment_kwargs=dict(flat_observation=True))
      env = DMControlEnv(env)
      env = FlattenDictWrapper(env, ['observations'])
      ...