Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix camera coord frames for all cameras and all robots for hw and sim #720

Merged
merged 15 commits into from
Sep 26, 2017
Merged

fix camera coord frames for all cameras and all robots for hw and sim #720

merged 15 commits into from
Sep 26, 2017

Conversation

fmessmer
Copy link
Member

@fmessmer fmessmer commented Sep 15, 2017 •
edited

fixes #669
fixes ipa320/cob4#553
requires https://github.com/ipa320/cob_common/pull/233

includes #715

@ipa-nhg @cagbal @ipa-fmw @ipa-rmb FYI

This was referenced Sep 15, 2017
Merged
Closed
Open
@fmessmer
Copy link
Member Author

fmessmer commented Sep 15, 2017 •
edited

in 51439d9 I added a nodelet_manager for simulation!
this was necessary as those cameras use the nodelet-variant of image_flip in the respective components-launch file...however, in simulation there was no nodelet_manager running so far...as this was part of the hw-driver...now we have all the X_upright/X topics in simulation, too

@fmessmer fmessmer changed the title fix camera coord frames for asus and zr300 on cob4-7 fix camera coord frames for all cameras and all robots for hw and sim Sep 18, 2017
@fmessmer
Copy link
Member Author

fmessmer commented Sep 20, 2017 •
edited

ToDo:

  • headcam coordinate system for cob4-12 needs to be updated, too
    (i.e. final comparison of cob_robots bs. unity_robots)
  • verify sick_3dcs and usb_cam topics and links with Hardware, e.g. tlabs

floweisshardt
floweisshardt previously approved these changes Sep 21, 2017
View reviewed changes
Copy link
Member

@floweisshardt floweisshardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested on cob4-12: OK

@fmessmer
Copy link
Member Author

fmessmer commented Sep 21, 2017 •
edited

I consider this final now
I've tested this on hardware (cob4-7 and cob4-10) as well as in simulation

Here are some screenshots of the live data from the robots:
cob4-7
head_cam
head_cam_live_data

sensorring (asus)
sensorring_live_data

torso_down (r200)
torso_down_live_data

torso_left (zr300)
torso_left_live_data

torso_right (zr300)
torso_right_live_data

cob4-10
head_cam
head_cam_live_data

sensorring_front (sick_3dcs)
sick_3dcs_live_data

sensorring_back (asus)
sensorring_back_live_data

torso_down (r200)
torso_down_live_data

torso_left (asus)
torso_left_live_data

torso_right (asus)
torso_right_live_data

@fmessmer
Copy link
Member Author

I've opened an issue for the wrong transformations in the sick_3dcs PointCloud here: https://github.com/SICKAG/sick_visionary_t/issues/19

fmessmer
fmessmer commented Sep 21, 2017
View reviewed changes
cob_bringup/components/cam3d_visionary_t.launch
<arg name="camera_name" value="$(arg name)"/>
<arg name="pointcloud_in" value="/$(arg name)/depth/points"/>
<arg name="pointcloud_out" value="/$(arg name)_upright/depth/points"/>
<!-- Sick3DCS does not have color image, and image flip cannot process images that are not CV_8UC3 -->
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ipa-rmb FYI
I don't know whether this is intended...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not care about that camera at the moment. Image flip would need to be extended to handle other types of images as well, the current code fpr 8UC3 images is speed optimized and not general. We can later add a slower general fall back solution for other image types if anyone needs that at all.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a new issue for this: ipa320/cob_perception_common#85

floweisshardt
floweisshardt approved these changes Sep 21, 2017
View reviewed changes
Copy link
Member

@floweisshardt floweisshardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

except for the flip_color_image: false for cob4-10, I'm OK with this PR.

cob_hardware_config/robots/cob4-10/config/image_flip_params_sensorring_cam3d_front.yaml
@@ -12,7 +12,7 @@ reference_frame: base_link

# flip color image
# bool
flip_color_image: true
flip_color_image: false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why false?

Copy link
Member Author

@fmessmer fmessmer Sep 21, 2017 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sick_3dcs does not have a color_image...@ipa-rmb @ipa-josh
and image_flip complained about not being able to flip the depth/image topic (see #720 (comment))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok fine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also fine with us, see comment above.

@fmessmer
Copy link
Member Author

@ipa-rmb @ipa-fmw
why do we actually need to flip the point cloud? it looks correct in rviz anyway...
does any node processing the point cloud depend on it being upright?
if this is not the case, we might safe some computational resources if we do not flip the point cloud, right?

@ipa-rmb
Copy link
Contributor

ipa-rmb commented Sep 25, 2017

Yes, because the pixel matrix should be the same for the color image and the point cloud. If I move pixels on the color image to a different location, then I would expect to find the corresponding XYZ pixel at the same pixel coordiates that I used in the color image. That is why the point cloud needs to be re-arranged as well.

@ipa-rmb
Copy link
Contributor

ipa-rmb commented Sep 25, 2017

Transforms and images for cob4-10 look fine.

@ipa-rmb
Copy link
Contributor

ipa-rmb commented Sep 25, 2017

Can/shall I merge?

@fmessmer fmessmer mentioned this pull request Sep 26, 2017
Open
@fmessmer
Copy link
Member Author

As I opened issues for all remaining issues, I'll merge all related PRs for now...
...in case anything is not as expected with this version, it needs to be investigated in a separate round...

@fmessmer fmessmer merged commit d17d14e into ipa320:indigo_dev Sep 26, 2017
@fmessmer fmessmer deleted the fix_camera_coord_frames branch September 26, 2017 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@floweisshardt floweisshardt floweisshardt approved these changes

@ipa-rmb ipa-rmb ipa-rmb approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

review camera coordinate frames
4 participants
@fmessmer @ipa-rmb @floweisshardt @benmaidel