Action Space Shaping in Deep Reinforcement Learning

Experiment code for the paper "Action Space Shaping in Deep Reinforcement Learning".

Testing different ways of modifying action spaces in various environments, and how that affects the learning performance. Long story short: Removing some "unnecessary" actions can be crucial for learning. Continuous actions should be discretized, but converting multi-discrete spaces to discrete is not too helpful.

Note: This does not include Starcraft 2 experiment code as of yet, as it is part of ongoing research. The IMPALA implementation used for that is based on Deepmind scalable-agents code.

Requirements

See requirements.txt for most requirements. Additionally obstacle-tower-env for running ObstacleTower experiments.

Running experiments

In root directory, run ./scripts/run_all.sh. If all goes well, this should create experiments directory with bunch of results.

Plotting

After running above experiments, run following to create figures shown in paper:

mkdir -p figures
python3 plot_paper.py experiments figures

If you lack some of the experiment files / plotting crashes in specific game, comment out corresponding lines in plot_paper.py main function main(args) (near the end of the file).