[Documentation]  

Deoxys Vision Utils

A package to use cameras in ros-independent manner and with deoxys_control (to move the robot to collect apriltag images)

Installation

1. Setup [deoxys_control](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys) (This usually should be setup)
2. Activate the conda environment (Usually should be deoxys_base)
3. Navigate into deoxys_vision folder
4. pip install -e .

Setup

Step 1: Set up the AprilTag on the Panda arm as shown

Step 2: Connect Spacemouse

Usage: Eye-to-Hand Calibration

This is to calibrate the fixed-base camera. This assumes that a marker mount is attached to the robot’s end-effector, and the marker is default to be AprilTag in our lab

Step 1: Record robot joints

Run the script:

python camera_calibration/record_robot_joints.py

The robot will run with OSC controller, with low-impedance value. In that case, it’s easy to move the arm around and record the desired joint values. For recording, press the grasping button of SpaceMouse (one of the buttons on the side). When you want to finish the process, press the other button of SpaceMouse which will terminate the process and let you decide 1) whether or not save the recorded joints, and 2) specify the file name to record the joints.

In order to see which joints are good for detecting tags, we suggest opening up the visualization of camera (and leave it running):

python scripts/deoxys_camera_node.py --camera-ref rs_0 --use-rgb --use-depth --use-rec --visualization --eval

Step 2: Replay robot joints and compute the extrinsics (the transformation of camera from the robot base frame)

python camera_calibration/hand_eye_calibration.py --camera-ref rs_0 --calibration-type eye-to-hand --config-filename calibration_joints.json

There are two useful options for running this script. One is --use-saved-images, which will use the previously recorded images without actually running the robot. Another one is --debug, which will show some detailed information of the calibration process.

Step 3: Saved config file

The calibrated value will be saved into the file camera_{ID}_extrinsics.json in the default folder ~/.deoxys_vision/calibration.

Step 4: (Optional step)

To test if the calibration was successful use the following script. Ensure that the calibrated value (the extrinsic matrix) from camera_{ID}_extrinsics.json is being used

python scripts/test_calibration.py --camera-ref rs_0 --calibration-method horaud

Usage: Eye-in-Hand Calibration

TODO