Thin_Obj_Bin_Picking

This is a ROS package presents a perception-to-manipulation system for picking thin objects from clutter. The manipulation consists of four steps: detecting the object of instance, approaching the target overhead, descending till the fingertip contacts the object, tilting to adjust the final grasp pose. Object detection is implemented on Mask R-CNN, a deep neural network for instance segmentation, while descending and tilting are implemented with tactile sensors.

Prerequisites

Hardware

1. Universal Robot UR10
2. Robotiq 140mm Adaptive parallel-jaw gripper
3. RightHand Labs's TakkStrip
4. Arduino
5. Realsense SR300

Software

1. ROS Kinetic
2. Driver for UR10 robot arms from universal robots
3. Universal Robot package for ROS Kinetic
4. MoveIt!
5. Robotiq ROS package
6. Mask R-CNN

.ipynb files can be run in jupyter notebook. Other requirements please check carefully in Mask R-CNN repository.

Get started

The following instructions will help you build up the software step by step.

1. Follow the tutorial in Universal Robot package for ROS Kinetic and Robotiq ROS package to set up hardware properly.

2. Run Realsense SR300 camera in ROS. See link.

3. Connect Arduino and tactile sensor, output the sensor readings in ROS. See link.

4. Setup frames:

   cd scripts
   

   python frame_transform.py 
   

5. Open a terminal, run object detection:

   cd samples
   

   jupyter notebook
   

   Open instance_segmentation.ipynb. For loading BLISTER_MODEL_PATH, please refer to here.

6. Open another terminal, run manipulation:

   cd scripts
   

   jupyter notebook
   

   Open thin_object_bin_pick_mani.ipynb

Author

Zhekai Tong (ztong@connect.ust.hk) and Tierui He (theae@connect.ust.hk)