Skip to content
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Motion Planning Benchmarks

for the RSS 2013 submission

Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization
John Schulman, Jonathan Ho, Alex Lee, Ibrahim Awwal, Henry Bradlow and Pieter Abbeel

This is a collection of scripts that runs planners--trajopt, OMPL/MoveIt, and CHOMP--on various testing scenes in various configurations. All testing scenes use the PR2.

The benchmarks are structured as a suite of problem sets. Each problem set is defined as a collection of DOF of the robot to plan, an environment description, and a set of start/goal joint pairs, each of which constitutes a single planning problem. A suite of problem sets is a set of configurations of planners (planner type and arguments) and a list of problem sets-- running a suite will run each of the planners on all of the configurations.


The benchmarking code depends on the following packages:

  • OpenRAVE, at least version 0.8 (see

  • Various Python packages: NumPy, YAML, JSON

Before running the benchmarks, you must install the planners:

  • trajopt, the planner described in this paper (see

  • ROS, MoveIt (see

    sudo apt-get install ros-groovy-moveit-full ros-groovy-moveit-full-pr2


    Note that the CHOMP implementation we use is not publicly available. If you do have access to it, however, then make sure that the orcdchomp Python package is available in your $PYTHONPATH.

Note that if any of these planners is not available, then it's still possible to run the benchmarks excluding the unavailable ones.

Trajopt and CHOMP require no pre-benchmarking setup, but MoveIt/OMPL benchmarks need the MoveIt ROS services to be running beforehand. To do this:

roscore &
roslaunch benchmark_scripts/moveit_without_retimer.launch

(note that we run MoveIt with the trajectory retimer disabled due to stability issues)

We also recommend setting the following environment variable:


to suppress excessive logging output from trajopt.

Running a testing suite

We provide the following suite files, which correspond to the experiments described in the paper:

  • problem_sets/suite_rightarm.yaml: arm planning problems for trajopt, OMPL, and CHOMP
  • problem_sets/suite_fullbody.yaml: full-body planning problems for trajopt and OMPL

(If you want to disable planner configurations for any reason, for example due to the unavailability of CHOMP, simply remove or comment out the appropriate entries under the configurations item in the YAML file.)

To run a suite SUITE_FILE.yaml and save the planning results to RESULTS_FILE.pkl, execute the following command from the root directory of this package:

python benchmark_scripts/ SUITE_FILE.yaml -o RESULTS_FILE.pkl

To analyze results in RESULTS_FILE.pkl, run:

python benchmark_scripts/ --summarize=RESULTS_FILE.pkl

An example analysis output will look like the following:

Problem set: bookshelves_rightarm.yaml, num problems: 45

Problem set: countertop_rightarm.yaml, num problems: 36

Each paragraph shows the results in CSV format for a particular problem set in the suite. The row names mean the following:

  • success_frac: Fraction of problems solved with valid, collision-free trajectories
  • avg_abs_path_len: Average joint-space path length among successful problems
  • avg_normed_path_len: Average normalized joint-space path length among successful problems. The normalized length of a trajectory for a single problem is defined as its absolute length divided by the minimum length for that problem across all planner configurations.
  • avg_time: Average time for planner to finish, for both successful and unsuccessful problems

Running individual problem sets

If you want to run an individual planning problem set (note that this is what does under the hood for each configuration/problemset pair):


Provided arm planning problem set files are

  • problem_sets/bookshelves_rightarm.yaml
  • problem_sets/countertop_rightarm.yaml
  • problem_sets/industrial_rightarm.yaml
  • problem_sets/industrial_rightarm2.yaml
  • problem_sets/tunnel_rightarm.yaml

and provided full-body planning problem set files are

The supported planners are trajopt, chomp, and ompl. Extra options can be provided to configure the behavior of the planners. For details, run

python benchmark_scripts/ --help
You can’t perform that action at this time.