Evaluation
We recommend to perform experiments with the Prost planner by using Prost Lab, an experiment package that is based on the Lab toolkit.
We believe that the easiest way to use Prost Lab is by installing it in a virtual environment. You can do so for every experiment (allowing you to use different Prost Lab versions in different experiment) or once for all experiments. We describe the latter approach in the following and assume that you have already installed the Prost planner in the directory /path/to/prost and rddlsim in the directory /path/to/rddlsim as described in the installation instructions.
To set up the virtual environment for Prost Lab in the directory /path/to/prostlab-venv, perform the following steps:
-
cd /path/to/prostlab-venv(switch to directory) -
python3 -m venv prostlab-venv(create virtual environment) -
source prostlab-venv/bin/activate(activate the virtual environment) -
pip install -U pip(upgrade pip) -
pip install prostlab(install prostlab)
If you want to install the latest development version and/or need to change Prost Lab itself, you can clone the Prost Lab repository and install it in the virtual environment:
-
git clone https://github.com/prost-planner/prostlab.git(clone the repo) -
cd prostlab(switch into prostlab directory) -
pip install --editable ./(install prostlab)
In both cases, you need to define two environment variables to perform experiments with Prost Lab:
- PROST_BENCHMARKS points to the testbed/benchmarks directory of your prost clone
- RDDLSIM_ROOT points to the root directory of rddlsim
If you haven't just set up Prost Lab as described in the previous point, you need to re-activate your virtual environment by switching to the directory containing the virtual environment for Prost Lab and by running source prostlab-venv/bin/activate. To run an experiment with Prost Lab, you need to set up a python script that uses Prost Lab. An example script can be found at testbed/experiments/issue-83/exp1.py. If you plan to perform experiments on the IPC benchmark suite that comes with Prost, we recommend to copy the file testbed/experiments/issue-83/ipc_benchmarks.py to your experiment folder and load the IPC benchmarks, e.g. by using the predefined benchmarks suites "IPC2011", "IPC2014", "IPC2018" or "IPC_ALL".