Benchmarking with different Motion Planners (STOMP, CHOMP, OMPL) #992

raghavendersahdev · 2018-07-20T14:50:00Z

Benchmarking with different Motion Planners (STOMP, CHOMP, OMPL)

This PR is a work done by Raghavender as a part of 2018 Google summer of code. As a part of this PR, benchmarking works for different motion planners (CHOMP, STOMP, OMPL) available in MoveIt. For a particular benchmark(start sates, planning scene, goal states, queries, etc.), different planners can be evaluated.

Checklist

addition of yaml files and launch files for including STOMP and CHOMP planners in environments with and without obstacles.
fixing the existing benchmarking code to make it work with STOMP and CHOMP.

mamoll · 2018-07-20T19:19:50Z

moveit_ros/benchmarks/examples/demo_obstacles.yaml

+        - plugin: chomp_interface/CHOMPPlanner
+          planners:
+            - CHOMP
+        - plugin: stomp_moveit/StompPlannerManager


maybe stomp_moveit should be renamed to stomp_interface to be consistent with OMPL and CHOMP?

I'm guessing this is tied by the package name

yes this is the package name: stomp_moveit

My point was that perhaps the package should be renamed for consistency. This can be done later, if others agree.

i agree it would be nice

davetcoleman · 2018-07-20T19:43:43Z

moveit_ros/benchmarks/examples/demo2.yaml

@@ -27,3 +27,4 @@ benchmark_config:
        - plugin: my_plugin_ns/my_plugin
          planners:
            - MyPlanner
+


please don't introduce changes like this into PRs since nothing else in this file was modified

sure this was my mistake, removed this change

davetcoleman · 2018-07-20T19:45:20Z

moveit_ros/benchmarks/src/BenchmarkExecutor.cpp

@@ -532,6 +532,9 @@ bool BenchmarkExecutor::plannerConfigurationsExist(const std::map<std::string, s
    planning_interface::PlannerManagerPtr pm = planner_interfaces_[it->first];
    const planning_interface::PlannerConfigurationMap& config_map = pm->getPlannerConfigurations();

+    if (pm->getDescription().compare("stomp") || pm->getDescription().compare("chomp"))


what is this for? add comment inline

added comments explaining this, I put this here because for STOMP and CHOMP, the benchmarking was throwing an error as this line for CHOMP/STOMP returns a size of 0 for the config_map parameter which was causing an error. After putting this check, everything worked fine.

mamoll

Ready to be merged once one small comment is updated.

mamoll · 2018-07-22T22:43:20Z

moveit_ros/benchmarks/examples/demo_obstacles.yaml

+        - plugin: chomp_interface/CHOMPPlanner
+          planners:
+            - CHOMP
+        - plugin: stomp_moveit/StompPlannerManager


My point was that perhaps the package should be renamed for consistency. This can be done later, if others agree.

mamoll · 2018-07-22T22:47:18Z

moveit_ros/benchmarks/src/BenchmarkExecutor.cpp

@@ -532,6 +532,11 @@ bool BenchmarkExecutor::plannerConfigurationsExist(const std::map<std::string, s
    planning_interface::PlannerManagerPtr pm = planner_interfaces_[it->first];
    const planning_interface::PlannerConfigurationMap& config_map = pm->getPlannerConfigurations();

+    // if the planner is chomp or stomp skip this function and return true for checking planner configurations for the
+    // planning group other wise an error occurs, this is kind of a trick which works well


Even with other wise replaced with otherwise, I still can't parse this. Why is this a trick? Perhaps the issue that with OMPL, a specific planning algorithm needs to be defined for a planning group, whereas with STOMP and CHOMP this is not necessary.

changed comment

raghavendersahdev · 2018-07-25T19:51:18Z

Sorry, I did something bad here while rebasing, accidentally rebased with the wrong local branch (raghavendersahdev/moveit kinetic branch) rather than the actual remote ros-planning/moveit kinetic branch, I am trying to fix it.

So at the end I basically merged the commits from this PR into the current one. But I am trying to fix this using git reflog

davetcoleman · 2018-07-27T20:56:08Z

its not clear if i should review this

raghavendersahdev · 2018-07-27T22:32:26Z

@davetcoleman I had a question for you, is it possible to undo a rebase after I already pushed it (I think it is kind of complicated coz I deleted my local branch on my laptop) ?

I actually wanted to rebase my raghavendersahdev/moveit benchmarking branch with the ros-planning/moveit kinetic branch ...But accidentally I rebased this with my own current raghavendersahdev/moveit kinetic branch.....So what ended up happening was => the wrong rebase merged the commits from my kinetic branch which also has a PR#987 open currently......

This specifc PR has only 6 commits but the branch which I used to rebase this with had 10 commits so I ended up with 16 commits in this PR.

Should I get rid of this PR and open a new one ? .... I am Sorry for the confusion pertaining to this specific PR.

davetcoleman · 2018-07-27T22:51:16Z

you can branch off of kinetic-devel then cherry-pick the relevant commits back on top of that, and it should work out

raghavendersahdev · 2018-07-27T23:06:37Z

ok, thanks Dave, I will read up on git cherry-pick and try to get rid of the 10 non-relevant commits in my raghavendersahdev/moveit benchmarking branch to only have the original 6 commits in the benchmarking branch

… STOMP and CHOMP

raghavendersahdev · 2018-07-29T03:51:11Z

Hi @davetcoleman , I have fixed this PR and removed the redundant commits added due to my wrong rebase earlier. This PR is now ready for review and Travis passes.

davetcoleman · 2018-07-30T16:34:13Z

moveit_ros/benchmarks/examples/demo_obstacles.yaml

+# of motion plan queries and start states in the Kitchen scene.
+
+# Five planners from two different plugins are run a total of 50 times each, with a
+# maximum of 10 seconds per run.  Output is stored in the /home/benchmarks directory.


Output is stored in the /home/benchmarks directory.

This is not true, right? Line 18 has a different location

yes, you are right, fixed!

davetcoleman · 2018-07-30T16:34:59Z

moveit_ros/benchmarks/examples/demo_panda_all_planners.yaml

+# of motion plan queries and start states in the Kitchen scene.
+
+# Five planners from two different plugins are run a total of 50 times each, with a
+# maximum of 10 seconds per run.  Output is stored in the /home/benchmarks directory.


same directory issue

davetcoleman · 2018-07-30T16:36:24Z

moveit_ros/benchmarks/src/BenchmarkExecutor.cpp

+    // planning group otherwise an error occurs, because for OMPL a specific planning algorithm needs to be defined for
+    // a planning group, whereas with STOMP and CHOMP this is not necessary
+    if (pm->getDescription().compare("stomp") || pm->getDescription().compare("chomp"))
+      return true;


this should be continue not return... in case OMPL is also in the list of planner interfaces to check

* addition of launch files and yaml files for benchmarking to work with STOMP and CHOMP * trick to make CHOMP and STOMP work with the existing benchmarks * changed default output directory in yaml files * applied clang format * addressed PR requested changes * fixed comment message * fixed some bugs in comments, addressed PR requested changes * q

davetcoleman · 2018-08-01T00:51:44Z

cherry picked to melodic

…eit#992) * addition of launch files and yaml files for benchmarking to work with STOMP and CHOMP * trick to make CHOMP and STOMP work with the existing benchmarks * changed default output directory in yaml files * applied clang format * addressed PR requested changes * fixed comment message * fixed some bugs in comments, addressed PR requested changes * q

Co-authored-by: Jafar Abdi <cafer.abdi@gmail.com> Co-authored-by: Tyler Weaver <tyler@picknik.ai>

mamoll reviewed Jul 20, 2018

View reviewed changes

davetcoleman reviewed Jul 20, 2018

View reviewed changes

mamoll approved these changes Jul 22, 2018

View reviewed changes

raghavendersahdev force-pushed the benchmarking branch from b236e84 to 3165b80 Compare July 25, 2018 19:45

raghavendersahdev added 6 commits July 28, 2018 22:44

addition of launch files and yaml files for benchmarking to work with…

dcce85b

… STOMP and CHOMP

trick to make CHOMP and STOMP work with the existing benchmarks

e6c3f0d

changed default output directory in yaml files

f7b59b3

applied clang format

3afadbd

addressed PR requested changes

b3b0c02

fixed comment message

ed9bdf6

raghavendersahdev force-pushed the benchmarking branch from 3165b80 to ed9bdf6 Compare July 29, 2018 02:49

davetcoleman requested changes Jul 30, 2018

View reviewed changes

raghavendersahdev added 2 commits July 31, 2018 01:14

fixed some bugs in comments, addressed PR requested changes

09fdd46

q

edb7d1e

davetcoleman approved these changes Aug 1, 2018

View reviewed changes

davetcoleman merged commit e4937c4 into moveit:kinetic-devel Aug 1, 2018

JafarAbdi added a commit to JafarAbdi/moveit that referenced this pull request Mar 24, 2022

Doxygen support (moveit#992)

37699f8

Co-authored-by: Jafar Abdi <cafer.abdi@gmail.com> Co-authored-by: Tyler Weaver <tyler@picknik.ai>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmarking with different Motion Planners (STOMP, CHOMP, OMPL) #992

Benchmarking with different Motion Planners (STOMP, CHOMP, OMPL) #992

raghavendersahdev commented Jul 20, 2018

mamoll Jul 20, 2018

davetcoleman Jul 20, 2018

raghavendersahdev Jul 21, 2018

mamoll Jul 22, 2018

davetcoleman Jul 27, 2018

davetcoleman Jul 20, 2018

raghavendersahdev Jul 21, 2018

davetcoleman Jul 20, 2018

raghavendersahdev Jul 21, 2018

mamoll left a comment

mamoll Jul 22, 2018

mamoll Jul 22, 2018

raghavendersahdev Jul 24, 2018

raghavendersahdev commented Jul 25, 2018 •

edited

Loading

davetcoleman commented Jul 27, 2018

raghavendersahdev commented Jul 27, 2018 •

edited

Loading

davetcoleman commented Jul 27, 2018

raghavendersahdev commented Jul 27, 2018

raghavendersahdev commented Jul 29, 2018

davetcoleman Jul 30, 2018

raghavendersahdev Jul 31, 2018

davetcoleman Jul 30, 2018

raghavendersahdev Jul 31, 2018

davetcoleman Jul 30, 2018

raghavendersahdev Jul 31, 2018

davetcoleman commented Aug 1, 2018

Benchmarking with different Motion Planners (STOMP, CHOMP, OMPL) #992

Benchmarking with different Motion Planners (STOMP, CHOMP, OMPL) #992

Conversation

raghavendersahdev commented Jul 20, 2018