cv2.CALIB_HAND_EYE_TSAI

[missFuture]

Hi,when i calibrate the realsense and kinova robot, i found these questions. please give me some advices
backgroud:
(1)launch the realsense
(2)launch the robot
(3)launch the kinova moveit
(4)start aruco and easy_handeye

software
python2.7, ubuntu16.04,opencv-python 4.2.032

rosdep error

here is my launch file(I disable automatic movement):

<launch>
    <arg name="namespace_prefix" default="kinova_d435_handeyecalibration" />
 
    <!--<arg name="robot_ip" doc="The IP address of the UR5 robot" />-->
 
    <arg name="marker_size" value="0.1" />
    <arg name="marker_id" value="14" />
 
    <!--<arg name="camera_frame" default="camera_rgb_optical_frame"/>-->
    <!--<arg name="reference_frame" default="camera_rgb_optical_frame"/>-->
    <!--<arg name="camera_marker" default="camera_link"/>-->
    <!-- start the Kinect -->
    <!--<include file="$(find realsense2_camera)/launch/ggcnn_points.launch" >-->
        <!--&lt;!&ndash;<arg name="depth_registration" value="true" />&ndash;&gt;-->
    <!--</include>-->
 
    <!-- start ArUco -->
    <node name="aruco_tracker" pkg="aruco_ros" type="single">
        <remap from="/camera_info" to="/camera/color/camera_info" />
        <remap from="/image" to="/camera/color/image_raw" />
        <param name="image_is_rectified" value="true"/>
        <param name="marker_size"        value="$(arg marker_size)"/>
        <param name="marker_id"          value="$(arg marker_id)"/>
        <param name="reference_frame"    value="camera_color_frame"/>
        <param name="camera_frame"       value="camera_color_frame"/>
        <!--<param name="camera_frame"       value="_color_optical_frame"/>-->
        <param name="marker_frame"       value="camera_marker" />
 
    </node>
 
     <!--start the robot -->
    <!--<include file="$(find kinova_bringup)/launch/kinova_robot.launch">-->
    <!--</include>-->
    <!--<include file="$(find j2n6s300_moveit_config)/launch/j2n6s300_demo.launch">-->
    <!--</include>-->
 
    <!-- start easy_handeye -->
    <include file="$(find easy_handeye)/launch/calibrate.launch" >
        <arg name="namespace_prefix" value="$(arg namespace_prefix)" />
        <arg name="eye_on_hand" value="true" />
 
        <arg name="tracking_base_frame" value="camera_color_frame" />
        <arg name="tracking_marker_frame" value="camera_marker" />
        <arg name="robot_base_frame" value="j2n6s300_link_base" />
        <arg name="robot_effector_frame" value="j2n6s300_link_6" />
        <!--disable automatic robot movement-->
        <arg name="freehand_robot_movement" value="true" />
        
        <arg name="robot_velocity_scaling" value="0.5" />
        <arg name="robot_acceleration_scaling" value="0.2" />
    </include>
 
</launch>

result:
i can get the moveit GUI and rviz_easy_handeye.config
there is an error when i launch my launch file named "kinova_realsense.launch".
i have tried some version of opencv-python, but error always exists.beside
I can not get the GUI of rqt_easy_handeye.perspective and rqt_easy_hand__Hand-eye Calibration automatic movement.

some picture

error picture

question
(1)help me about the cv problem(cv2.CALIB_HAND_EYE_TSAI)
(2)GUI of rqt_easy_handeye.perspective and rqt_easy_hand__Hand-eye Calibration automatic movement
(3)when i run rosrun rqt_easy_handeye rqt_easy_handeye to launch rqt_easy_handeye. error like that

Thanks a lot!

[lyh458]

Here are some of my suggestions:

1. for the rosdep error, try to install transform3d by command pip install transforms3d
2. for the AttributeError, I think it's similar to this issue, you can try this command python -m pip install opencv-contrib-python
3. for your last error, I think you can not simply run rosrun rqt_easy_handeye rqt_easy_handeye without arg input. I have the same error with the same command as you even though I have used this repo successfully. So this is an irrelevant error for you to use this repo.

[marcoesposito1988]

Thanks for the help @lyh458!

1. this is a mistake on my side, I mistyped the rosdep name. I will fix this
2. this is because rosdep is installing the OpenCV Python bindings, but on 16.04 version 2.4.9 is installed, which doesn't include the hand-eye calibration algorithms. If you can't upgrade to 18.04 and melodic yet, lyh458's solution will work because it installs a PIP package which bundles a very recent version of OpenCV
3. lyh458 is also correct in this regard. However, if you ran the launch file you posted, that file will set all the required arguments in the ROS parameter server. So after running your launch file, you can start the node with:

ROS_NAMESPACE=kinova_d435_handeyecalibration_eye_on_hand rosrun rqt_easy_handeye rqt_easy_handeye

(I am writing this off the top of my head, so the namespace above may not be completely correct in your case; you can find the correct one with rosparam list)

[yangbenbo]

Thanks for the help @lyh458 !
I had a cv problem(cv2.CALIB_HAND_EYE_TSAI) yesterday.
I find the path of python in ROS:(/opt/ros/kinetic/lib/python2.7/dist-packages) is in front of many other path of python, but the cv2.so in ROS is not approprite because of lack CALIB_HAND_EYE_TSAI.(I have use pip3 install opencv-python python -m pip install opencv-contrib-python to install module and it's installed in ~/.local/lib/python2.7 which is after the path of python in ROS)

I have to use the code below to find the correct of cv2.
import sys
sys.path.remove('/opt/ros/kinetic/lib/python2.7/dist-packages')
import cv2
sys.path.append('/opt/ros/kinetic/lib/python2.7/dist-packages')

I'd like to know is there any other solution. Thanks!

My environment:
ubuntu16, ros-kinetic

[jackie-wayland]

@marcoesposito1988 same issue on ubuntu 18.04 and ROS melodic

[missFuture]

@lyh458 Thanks for your kind help, follow your instructions, i have solved these problems
Q1.pip install transforms3d
Q2.python -m pip install opencv-contrib-python, and then follow @yangbenbo 's advice, this problem is done(thank you).
Q3.after the two qusetion, Q3 is normal automaticaly.I just want to test the single node to debug(rqt_easy_handeye), thanks for your advice, it works.

[ZZ-111]

@lyh458 Thanks for your kind help, follow your instructions, i have solved these problems
Q1.pip install transforms3d
Q2.python -m pip install opencv-contrib-python, and then follow @yangbenbo 's advice, this problem is done(thank you).
Q3.after the two qusetion, Q3 is normal automaticaly.I just want to test the single node to debug(rqt_easy_handeye), thanks for your advice, it works.

I also have the following problems.
"I can not get the GUI of rqt_easy_handeye.perspective and rqt_easy_hand__Hand-eye Calibration automatic movement."

Can you tell me how to solve them?Looking forward to your help.

[baolin-hri]

I have the same problem and is there any progress.

[marcoesposito1988]

The problem should be solved with the inclusion of python-opencv among the catkin dependencies. However, this requires you to run rosdep as specified in the README:

cd <your catkin workspace>
rosdep install -yir --from-paths src

Can you try this step and report if it solves the problem?

[baolin-hri]

Thanks for your reply.
I have solved this problem by updating ubuntu 16.04 to 18.04.

[marcoesposito1988]

Closing until further bug reports.

[therealdevnull]

I think I'm seeing the same problem with the following environment:

Noetic / Ubuntu 20.04.2 / easy_handeye v0.4.2

E: Unable to locate package python-opencv
ERROR: the following rosdeps failed to install
pip: command [sudo -H python3 -m pip install -U transforms3d] failed
pip: Failed to detect successful installation of [transforms3d]
apt: command [sudo -H apt-get install -y python-opencv] failed
apt: Failed to detect successful installation of [python-opencv]

Following the advice found in this forum, I did this:

pip2 install transforms3d
python3 -m pip install opencv-contrib-python

That solved the transforms3d problem but not so for python-opencv:

rosdep install -iyr --from-paths src
executing command [sudo -H apt-get install -y python-opencv]
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package python-opencv
ERROR: the following rosdeps failed to install
  apt: command [sudo -H apt-get install -y python-opencv] failed
  apt: Failed to detect successful installation of [python-opencv]

Where am I going wrong?

I'm new to ROS so I'm sure it's something basic that I've got wrong but it's really not obvious to me.

Taking a step back, I can change the release of ROS or Ubuntu that I use so let me ask: Is there an ROS/ OS combination with which I might expect better results?

I've tried reverting to 18.04 but Noetic itself ran into install problems. Perhaps I should be using ROS 2 but the readme is specific about catkin which I believe doesn't work on ROS 2...

Any help getting easy_handeye running would be appreciated.

Thanks.

[marcoesposito1988]

Hi @therealdevnull,

most of the trouble is due to the switch from python 2 to python 3 that happened between ROS Melodic and Noetic. Apparently it broke some mechanisms in rosdep, upon which the setup procedure depends.

easy_handeye specifies transforms3d and the OpenCV python bindings as dependencies, and they are supposed to be installed automatically when you run rosdep.

The second error seems to be caused by the rosdep repository being outdated. The correct package on 20.04 is python3-opencv, but it tries to install python-opencv (which is no longer present in APT). I will report the error to rosdistro. You can solve the problem on your side by using APT manually: sudo apt install python3-opencv

The first error is weirder. transforms3d seems to be present on PyPI, so I don't know why it fails within rosdep. You installed it successfully on your own, but that was for python 2. You should be able to install it for python3 with pip3 install transforms3d

[marcoesposito1988]

The problem should be solved now with d8caff0. The proper solution is to add a conditional dependency on the python version.

[therealdevnull]

@marcoesposito1988 Thank you very much for the help.

For anyone else that this might benefit...

I ended up having to revert (again!) to 18.04 but this time with Melodic instead of Noetic. I had much better luck with this combination. I think the only issue I had to address was this: #78

Thanks again.