Skip to content
An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods - EWRL Workshop 2018
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
benchmarks Initial Commit Oct 1, 2018
bin
optimumrl
scripts
.gitignore
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE.md
README.md

README.md

Where Did My Optimum Go? Experiments.

This is the repository for the experiments seen in the EWRL 2018 Paper "Where Did My Optimum Go?":

https://arxiv.org/abs/1810.02525

Largely they are a modification of the code found in:

https://github.com/ikostrikov/pytorch-a2c-ppo-acktr

Modifications are mostly superficial to allow reading of configuration files for easier experiment running and distribution.

Example

You can run one of the ablation experiments for example via:

./bin/indexed_experiment --config_file ./benchmarks/configurations/continuous/a2c/continuous_suite.json --run_index 0 --ablation_config ./benchmarks/configurations/papers/hendersonromoff2018optimizer/ablation/sgd/momentum.json

Possible Issues

You may need to add the current directory to your pythonpath if you have problems running the experiment:

export PYTHONPATH=${PYTHONPATH}:.

Citation

If you find this useful, please cite our work:

@inproceedings{hendersonromoff2018optimizer,
  author    = {Peter Henderson and Joshua Romoff and Joelle Pineau},
  title     = {Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods},
  booktitle = {The 14th European Workshop on Reinforcement Learning (EWRL 2018)},
  year      = {2018}
}

Additionally, if you are relying on the codebase heavily please note the original codebase as well:

@misc{pytorchrl,
  author = {Kostrikov, Ilya},
  title = {PyTorch Implementations of Reinforcement Learning Algorithms},
  year = {2018},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/ikostrikov/pytorch-a2c-ppo-acktr}},
}

License

This repo is CC-BY-NC licensed, as found in the LICENSE file.

You can’t perform that action at this time.