gazebo_ros_openni_kinect plugin with roll + pitch orientation generates wrong depth image

[efernandez]

The problem is probably because of a wrong depth image coming from gazebo, as explained here:
http://answers.gazebosim.org/question/7005/gazebo_ros_openni_kinect-plugin-with-roll-pitch/

I'm asking here because it's related with a plugin tough.

[j-rivero]

There is a good explanation of the problem and a possible workaround in ros answers

[annamannucci]

Hi, I have the same problem. I'm using gazebo 4, ROS Indigo with Ubuntu 14.04. I would like to know if someone has tried to correct this bug.

[felixvd]

This is still a bug in Kinetic. I get incorrectly rotated point clouds. It seems like something is wrong with the coordinate systems, because moving the robot results in behavior like shown below.

In these pictures, the camera is looking down at the bins. The point cloud is not displayed at the correct place. When the robot moves to the goal pose of the second picture, the point cloud rotates in more than one axis:

Another shot with an extra movement:

And a close-up of the frames:

I assumed that the realsense camera URDF I am using is rotating the camera frames, and gazebo might not like that. The question posted by @j-rivero seems to imply as much. But removing all rotations from the camera's xacro so that the frames are oriented like this does not change the behavior:

I wondered if it had something to do with Gazebo connecting the sensor to the wrist link, whose parent frame is not oriented like the camera, but even taking out the rotation from that did not fix it. In the last picture, all the frames on wrist_link_2 have the same orientation, and the camera is looking towards the right at the robot and wall. The projection is still not correct.

Any ideas?

[vib2810]

Here is a possible fix:

There are 3 approaches

1. edit, recompile and replace the .so plugin file (Quite hard to compile)
2. Edit the baselink->camera transform
3. If 2 isn't possible due to the transform being published according to the URDF, a dummy link can be created and the frameName of the camera attached to this dummy link RPY(-pi/2, 0, -pi/2) wrt the initial camera frame

All these solutions can be found here:
https://answers.gazebosim.org/question/4266/gazebo-15-libgazebo_openni_kinectso-pointcloud-data-is-rotated-and-flipped/?answer=26388#post-id-26388
Code for my fix(Method 3) is also present in this thread