This is the official code, implemented by Jan Wöhlke, accompanying the ICRA 2021 paper Hierarchies of Planning and Reinforcement Learning for Robot Navigation by Jan Wöhlke, Felix Schmitt, and Herke van Hoof. The paper can be found here. The code allows the users to reproduce and extend the results reported in the paper. Please cite the above paper when reporting, reproducing or extending the results:
@inproceedings{woehlke2021hierarchies,
title = {Hierarchies of Planning and Reinforcement Learning for Robot Navigation},
author = {Jan W{\"o}hlke and Felix Schmitt and van Hoof, Herke},
year = {2021},
booktitle = {IEEE International Conference on Robotics and Automation}
}
This software is a research prototype, solely developed for and published as part of the publication cited above. It will neither be maintained nor monitored in any way.
Place the folder code where you want to execute the code.
Add the path of the folder code to your $PYTHONPATH environment variable.
You need a Python set-up with the following packages:
- python>=3.7.6
- numpy>=1.18.1
- scipy>=1.4.1
- torch==1.3.1
For the "Four Rooms" simple point-mass navigation experiments in Section V.C. / Figure 4 a) and b) in the paper you find the start scripts here:
Without "terrain":
With "terrain":
Run the scripts with: python [scriptname] --seed=[seed option]
There are different seed options to run the number random seeds used for the respective experiment:
- "s1" to "s10" : run an individual random seed
- not specified / other: run all ten seeds in sequence
You need to have Mujoco (version 2.0 - mujoco/200) installed and have a valid license key for it.
You need to install the following additional Python packages:
- gym==0.15.4 (install in editable mode: pip install -e)
- mujoco-py (our version: 2.0.2.10, https://github.com/openai/mujoco-py)
The start scripts for the experiments in Section V.D. / Figure 4 c) can be found here:
Before running the scripts, you need to fill the folder path of the assets folder, where the ant.xml file is located, of your gym installation as the mj_ant_path (where it says 'TO_BE_ENTERED' in the respective script).
Note that during execution the ant.xml will be modified constantly. In case the program does not terminate properly, it might remain in a modified state and needs to be restored from the generated ant_copy.xml file, in order to obtain correct results in subsequent simulations.
Run the scripts with: python [scriptname] --seed=[seed option]
There are different seed options to run the number random seeds used for the respective experiment:
- "s1" to "s10" : run an individual random seed
- not specified: run all ten seeds in sequence
You need to install the following additional Python packages:
- gym==0.15.4 (may already be done for the robotic maze navigation experiments)
- highway-env (https://github.com/eleurent/highway-env)
The start scripts for the experiments in Section V.E. / Figure 5 can be found here:
Run the scripts with: python [scriptname] --seed=[seed option]
There are different seed options to run the number random seeds used for the respective experiment:
- "s1" to "s10" : run an individual random seed
- not specified: run all ten seeds in sequence
Plot the test goal reaching probabilities in the "results_ ..." files of all the random seeds of the respective experiment of interest as mean +- standard error to arrive at the curves shown in the paper.
Hierarchies-of-Planning-and-Reinforcement-Learning-for-Robot-Navigation is open-sourced under the AGPL-3.0 license. See the LICENSE file for details.
For a list of other open source components included in Hierarchies-of-Planning-and-Reinforcement-Learning-for-Robot-Navigation, see the file 3rd-party-licenses.txt.