Skip to content
An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods - EWRL Workshop 2018
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
benchmarks Initial Commit Oct 1, 2018

Where Did My Optimum Go? Experiments.

This is the repository for the experiments seen in the EWRL 2018 Paper "Where Did My Optimum Go?":

Largely they are a modification of the code found in:

Modifications are mostly superficial to allow reading of configuration files for easier experiment running and distribution.


You can run one of the ablation experiments for example via:

./bin/indexed_experiment --config_file ./benchmarks/configurations/continuous/a2c/continuous_suite.json --run_index 0 --ablation_config ./benchmarks/configurations/papers/hendersonromoff2018optimizer/ablation/sgd/momentum.json

Possible Issues

You may need to add the current directory to your pythonpath if you have problems running the experiment:



If you find this useful, please cite our work:

  author    = {Peter Henderson and Joshua Romoff and Joelle Pineau},
  title     = {Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods},
  booktitle = {The 14th European Workshop on Reinforcement Learning (EWRL 2018)},
  year      = {2018}

Additionally, if you are relying on the codebase heavily please note the original codebase as well:

  author = {Kostrikov, Ilya},
  title = {PyTorch Implementations of Reinforcement Learning Algorithms},
  year = {2018},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{}},


This repo is CC-BY-NC licensed, as found in the LICENSE file.

You can’t perform that action at this time.