Mesh Filter tutorial with UR5 and Kinect simulated in Gazebo #558

Abishalini · 2020-11-20T12:06:15Z

Description

Mesh Filter tutorial for Melodic.
The mesh filter takes in a point cloud, tf of robot, and a URDF and publishes a modified point cloud with the point cloud subtracted that overlaps with the current robot state.
A separate Gazebo tutorial still needs to be made (Gazebo setup for this tutorial does not depend on ARM_moveit_config's auto-generated files)

Checklist

Required by CI: Code is auto formatted using clang-format
While waiting for someone to review your request, please consider reviewing another open pull request to support the maintainers

welcome · 2020-11-20T12:06:18Z

Thanks for helping in improving MoveIt and open source robotics!

davetcoleman

Thank for working on this Abi!

doc/mesh_filter/launch/test_gazebo.launch

doc/mesh_filter/mesh_filter_tutorial.rst

doc/mesh_filter/meshes/hokuyo.dae

tylerjw

When I try to test this my console is flooded with this error:

ros.moveit_ros_perception: TF Problem: Invalid argument passed to lookupTransform argument target_frame in tf2 frame_ids cannot be empty

Also, I can't seem to get the visualization of the point cloud working in rviz like it shows in the images and gif in the tutorial.

doc/mesh_filter/mesh_filter_tutorial.rst

doc/mesh_filter/launch/test_gazebo.launch

tylerjw · 2021-01-05T23:08:57Z

Here is my vcs status from the src directory in case it differs from yours:

$ vcs status
.........
=== ./geometric_shapes (git) ===
On branch noetic-devel
Your branch is up to date with 'origin/noetic-devel'.

nothing to commit, working tree clean
=== ./moveit (git) ===
On branch pr-fix_mesh_filter
Your branch is up to date with 'jafar/pr-fix_mesh_filter'.

nothing to commit, working tree clean
=== ./moveit_msgs (git) ===
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
=== ./moveit_resources (git) ===
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
=== ./moveit_tutorials (git) ===
On branch add_sensor
nothing to commit, working tree clean
=== ./moveit_visual_tools (git) ===
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
=== ./panda_moveit_config (git) ===
On branch melodic-devel
Your branch is up to date with 'origin/melodic-devel'.

nothing to commit, working tree clean
=== ./rviz_visual_tools (git) ===
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
=== ./universal_robot (git) ===
On branch melodic-devel
Your branch is up to date with 'origin/melodic-devel'.

nothing to commit, working tree clean

Here is my catkin config:

$ catkin config
-------------------------------------------------------------------------------------------------
Profile:                     default
Extending:        [explicit] /opt/ros/melodic
Workspace:                   /home/tyler/code/moveit/ws_moveit
-------------------------------------------------------------------------------------------------
Build Space:        [exists] /home/tyler/code/moveit/ws_moveit/build
Devel Space:        [exists] /home/tyler/code/moveit/ws_moveit/devel
Install Space:      [unused] /home/tyler/code/moveit/ws_moveit/install
Log Space:          [exists] /home/tyler/code/moveit/ws_moveit/logs
Source Space:       [exists] /home/tyler/code/moveit/ws_moveit/src
DESTDIR:            [unused] None
-------------------------------------------------------------------------------------------------
Devel Space Layout:          linked
Install Space Layout:        None
-------------------------------------------------------------------------------------------------
Additional CMake Args:       -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo
Additional Make Args:        None
Additional catkin Make Args: None
Internal Make Job Server:    True
Cache Job Environments:      False
-------------------------------------------------------------------------------------------------
Whitelisted Packages:        None
Blacklisted Packages:        None
-------------------------------------------------------------------------------------------------
Workspace configuration appears valid.
-------------------------------------------------------------------------------------------------

doc/mesh_filter/mesh_filter_tutorial.rst

davetcoleman · 2021-03-10T16:30:38Z

@Abishalini what's the status of this PR?

JStech · 2021-03-11T02:56:02Z

When I try to test this my console is flooded with this error:
ros.moveit_ros_perception: TF Problem: Invalid argument passed to lookupTransform argument target_frame in tf2 frame_ids cannot be empty

It took me a bit to track this down, but that's because the filter initializes the input optical frame with the empty string, and it doesn't get updated until it receives a depth map. However, it doesn't subscribe to the depth input until its output has a subscriber. So once you subscribe to /filtered/depth or /model/depth, the spam stops.

So, I don't think this is an error in this tutorial, but just undesirable behavior in the mesh filter itself. I'll look into it a bit more and see if I can fix it easily, otherwise I think we should just throttle that error.

JStech

Looks good! I mostly made some formatting suggestions.

The flood of transform errors is a MoveIt issue, and so I don't think you should work around it here--but thanks for trying!

I was able to build and run this as advertised. The necessary MoveIt PR has merged, so after removing the extra subscriber, I think this is good to go.

doc/mesh_filter/CMakeLists.txt

doc/mesh_filter/launch/mesh_filter.launch

doc/mesh_filter/launch/test_gazebo.launch

doc/mesh_filter/mesh_filter_tutorial.rst

doc/mesh_filter/src/filtered_depth_subscriber.cpp

davetcoleman · 2021-04-28T17:33:56Z

@Abishalini could we get the final fixes for this to be merged in? This feature came up in a meeting today.

v4hn

I added another review.

Because the whole perception pipeline uses the mesh_filter, there is a slight overlap there.
Maybe you could add a sentence to the perception pipeline tutorial saying that the mesh_filter (link to this new tutorial) will already exclude the robot model from the processed point data.

The flood of transform errors is a MoveIt issue, and so I don't think you should work around it here--but thanks for trying!
... The necessary MoveIt PR has merged

Which PR? Are the mentioned error messages gone?

v4hn · 2021-04-28T20:13:30Z

doc/mesh_filter/urdf/ur5_sensor.gazebo

@@ -0,0 +1,39 @@
+<?xml version="1.0"?>


Nothing in this file refers to UR. Also it's not just "sensor", but a camera model.
How about calling the file kinect_camera.gazebo?

why do you even specify the camera model in a separate file at all?

Renamed this file

v4hn · 2021-04-28T20:15:26Z

doc/mesh_filter/launch/mesh_filter.launch

+
+  <node pkg="moveit_tutorials" type="filtered_depth_subscriber_node" name="filtered_depth_subscriber_node" output="screen"/>
+
+  <include file="$(find moveit_tutorials)/doc/mesh_filter/launch/test_gazebo.launch"/>


includes between launch files in the tutorials do not work because the files are installed in different locations between devel and install workspaces. I found the same issue in another tutorial yesterday.

Given that there is no other reference to test_gazebo.launch, I would just include its content here.

Would $(dirname)/test_gazebo.launch work? I've not had a reason to use dirname before but it seems applicable here. Ref: bottom of this section

Would $(dirname)/test_gazebo.launch work?

Yes! I wonder how I managed to miss the introduction of that feature!
Last time I checked this website was before kinetic 😄 .
Thanks for pointing it out. I have to rewrite so many launch files now....

With this we can keep the separate files, but I would still propose to change the name. This is no test.
Something like ur5_gazebo.launch would work for me.

Also, I'm still no fan of using a different robot for this tutorial (it adds another dependency that will break at some point), but I don't know whether the panda gazebo simulation actually works at the moment.

I chnaged the name of the launch file to ur5_gazebo.launch

doc/mesh_filter/launch/test_gazebo.launch

doc/mesh_filter/CMakeLists.txt

Abishalini · 2021-04-29T04:42:59Z

The error messages mentioned by @tylerjw was resolved in PR moveit/moveit#2550

v4hn · 2021-04-29T18:54:15Z

@Abishalini I believe you messed up your PR branch. There are changes all over the place now.

v4hn · 2021-04-29T19:08:55Z

It's a bit hard to see at the moment because of all the unrelated commits right now, but I believe you also did not add the new package dependencies to the package.xml yet.
That is necessary assuming we can't use the panda for this tutorial for some reason (I can imagine a few...).

This reverts commit e76ed84.

v4hn · 2021-04-30T16:04:09Z

doc/mesh_filter/CMakeLists.txt

+install(DIRECTORY launch DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})
+install(DIRECTORY meshes DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})
+install(DIRECTORY urdf DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})
+install(DIRECTORY rviz DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})


According to the current layout all things except for the launch files should be installed to ${CATKIN_PACKAGE_SHARE_DESTINATION}/doc/<tutorial>.
So you don't have to do this weird dance with the additional parameter for the mesh in the xacro.

In theory we could also move all the launch files to this subfolder, but users likely expect them all in one place, so we don't.

Sorry for the confusion. 😕

The tutorial now works for both devel and install workspaces.

JStech

LGTM. There are a few files that don't have final newlines in them, if you're looking for one more thing to change :-P

v4hn · 2021-05-03T20:33:48Z

doc/mesh_filter/CMakeLists.txt

+install(DIRECTORY launch DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})
+install(DIRECTORY meshes DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})
+install(DIRECTORY urdf DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})
+install(DIRECTORY rviz DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION})


Currently, the install folder of moveit_tutorials looks like this:

❯ ls install/share/moveit_tutorials/ cmake doc launch package.xml

where the doc folder already contains subfolders for some tutorials with auxiliary data.
I would propose to leave meshes, urdf and rviz files there as well, only launch files are grouped together in one folder.
If you want, I can do these changes myself.

Incorporated the changes you recommended

v4hn · 2021-05-03T20:42:19Z

package.xml

+  <!-- The universal robot package is not released for melodic -->
+  <!-- Make sure to clone the package (melodic-devel branch) onto your workspace -->
+  <!-- <run_depend>ur_gazebo</run_time> -->
+  <!-- <run_depend>ur_description</run_time> -->


Why did you comment out the run_depends?
If the packages are in the same workspace, these should be fine, right?
And if they aren't then rosdep tells you exactly the same as the comments above 😃

Adding those dependencies fails CI

This has to be resolved either way, but I'll merge this now and create a corresponding issue.

v4hn · 2021-05-04T11:24:27Z

Thanks for your perseverance. 💐
I'm sure you hate me by now. :-)

welcome · 2021-05-04T11:25:02Z

Congrats on getting your first MoveIt pull request merged and improving open source robotics!

davetcoleman · 2021-05-04T17:33:49Z

Hurray! 🥇

v4hn · 2021-08-06T13:20:41Z

Hurray! 🥇

Not really anymore by now. This patch added gazebo_ros to our MoveIt dependencies which takes forever to download and is a mountain of data to install in our CI for no merit 😿
My main criticism here is that this is the only place in the whole MoveIt rosinstall workspace where gazebo is required and that it hardly justifies the overhead...

JafarAbdi self-assigned this Nov 20, 2020

Abishalini changed the title ~~Add sensor~~ Mesh Filter tutorial with UR5 and Kinect simulated in Gazebo Nov 20, 2020

JafarAbdi force-pushed the add_sensor branch from a752fa7 to 0a28b28 Compare November 20, 2020 17:16

JafarAbdi changed the base branch from melodic-devel to master November 20, 2020 17:18

davetcoleman reviewed Nov 23, 2020

View reviewed changes

doc/mesh_filter/launch/test_gazebo.launch Outdated Show resolved Hide resolved

doc/mesh_filter/mesh_filter_tutorial.rst Outdated Show resolved Hide resolved

doc/mesh_filter/meshes/hokuyo.dae Outdated Show resolved Hide resolved

Abishalini assigned tylerjw Dec 2, 2020

JafarAbdi mentioned this pull request Dec 7, 2020

Enable mesh filter moveit/moveit#2448

Merged

6 tasks

tylerjw reviewed Jan 5, 2021

View reviewed changes

doc/mesh_filter/mesh_filter_tutorial.rst Outdated Show resolved Hide resolved

doc/mesh_filter/launch/test_gazebo.launch Outdated Show resolved Hide resolved

tylerjw reviewed Jan 7, 2021

View reviewed changes

doc/mesh_filter/mesh_filter_tutorial.rst Outdated Show resolved Hide resolved

Abishalini requested a review from JStech March 10, 2021 17:50

JStech requested changes Mar 11, 2021

View reviewed changes

v4hn reviewed Apr 28, 2021

View reviewed changes

Abishalini added 10 commits April 29, 2021 15:24

Mesh Filter skeleton

a10a06d

Launch file in working condition - Need to fix sensor plugin

d2c930f

Fixed table sliding in Gazebo

3f2a607

Rectified Kinect Orientation

f663e4f

Start content on rst file

1d1d598

More documentation

e5e2252

Changes to documentation (as suggested by Dave)

9377017

Add tutorial to index.rst

5d472d3

Changes to documentation

6317a06

subscribe to /filtered/depth

0c6d64c

Abishalini added 2 commits April 29, 2021 15:24

Revert "subscribe to /filtered/depth"

85d695c

This reverts commit e76ed84.

Changes recommended by John

274bbfa

Abishalini force-pushed the add_sensor branch from b40caa4 to 274bbfa Compare April 29, 2021 22:25

Fix for install workspace

ca4fbac

v4hn reviewed Apr 30, 2021

View reviewed changes

Abishalini added 3 commits April 30, 2021 10:46

Add dependencies to package.xml

89b2129

Remove ur_description from package.xml

d265987

Remove ur_gazebo from package.xml

1a5156b

v4hn mentioned this pull request May 3, 2021

Improve perception tutorial #633

Merged

Rename sensor plugin file

715e055

JStech approved these changes May 3, 2021

View reviewed changes

Add newline at EOF

dcc6a56

v4hn reviewed May 3, 2021

View reviewed changes

Change layout in install space

22bbe7a

v4hn mentioned this pull request May 4, 2021

Breaking changes in Melodic release ros-industrial/universal_robot#448

Closed

fix line ending

66ce57c

v4hn approved these changes May 4, 2021

View reviewed changes

v4hn merged commit 13c4fc1 into moveit:master May 4, 2021

v4hn mentioned this pull request May 4, 2021

UR dependencies not declared in package.xml #638

Open

Abishalini deleted the add_sensor branch May 11, 2021 18:53

JStech mentioned this pull request May 11, 2021

Create tutorial for using mesh filter #524

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mesh Filter tutorial with UR5 and Kinect simulated in Gazebo #558

Mesh Filter tutorial with UR5 and Kinect simulated in Gazebo #558

Abishalini commented Nov 20, 2020 •

edited

welcome bot commented Nov 20, 2020

davetcoleman left a comment

tylerjw left a comment

tylerjw commented Jan 5, 2021 •

edited

davetcoleman commented Mar 10, 2021

JStech commented Mar 11, 2021

JStech left a comment •

edited

davetcoleman commented Apr 28, 2021

v4hn left a comment

v4hn Apr 28, 2021

Abishalini May 3, 2021

v4hn Apr 28, 2021

JStech Apr 29, 2021 •

edited

v4hn Apr 29, 2021

Abishalini Apr 30, 2021

Abishalini commented Apr 29, 2021 •

edited

v4hn commented Apr 29, 2021

v4hn commented Apr 29, 2021

v4hn Apr 30, 2021

Abishalini May 3, 2021

JStech left a comment

v4hn May 3, 2021

Abishalini May 4, 2021

v4hn May 3, 2021

Abishalini May 4, 2021

v4hn May 4, 2021

v4hn commented May 4, 2021

welcome bot commented May 4, 2021

davetcoleman commented May 4, 2021

v4hn commented Aug 6, 2021


		<node pkg="moveit_tutorials" type="filtered_depth_subscriber_node" name="filtered_depth_subscriber_node" output="screen"/>

		<include file="$(find moveit_tutorials)/doc/mesh_filter/launch/test_gazebo.launch"/>

Mesh Filter tutorial with UR5 and Kinect simulated in Gazebo #558

Mesh Filter tutorial with UR5 and Kinect simulated in Gazebo #558

Conversation

Abishalini commented Nov 20, 2020 • edited

Description

Checklist

welcome bot commented Nov 20, 2020

davetcoleman left a comment

Choose a reason for hiding this comment

tylerjw left a comment

Choose a reason for hiding this comment

tylerjw commented Jan 5, 2021 • edited

davetcoleman commented Mar 10, 2021

JStech commented Mar 11, 2021

JStech left a comment • edited

Choose a reason for hiding this comment

davetcoleman commented Apr 28, 2021

v4hn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JStech Apr 29, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Abishalini commented Apr 29, 2021 • edited

v4hn commented Apr 29, 2021

v4hn commented Apr 29, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JStech left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

v4hn commented May 4, 2021

welcome bot commented May 4, 2021

davetcoleman commented May 4, 2021

v4hn commented Aug 6, 2021

Abishalini commented Nov 20, 2020 •

edited

tylerjw commented Jan 5, 2021 •

edited

JStech left a comment •

edited

JStech Apr 29, 2021 •

edited

Abishalini commented Apr 29, 2021 •

edited