Skip to content

Home

Jump to bottom
PriyankaKhante edited this page Mar 30, 2018 · 30 revisions

Welcome to the hlpr_perception wiki!

Installing perception nodes

  1. Follow the Kinect 2 installation tutorial here. Follow all the required steps, do NOT follow the steps to install CUDA! This is an optional step that will have you download the wrong NVIDIA drivers, and will cause you to waste a day trying to figure out why your computer has locked you out. DON'T DO IT!
  • If you are using 14.04: There is a line under "Install OpenGL" that is (Ubuntu 14.04 only) sudo dpkg -i debs/libglfw3*deb; sudo apt-get install -f; sudo apt-get install libgl1-mesa-dri-lts-vivid (If the last command conflicts with other packages, don't do it.) THAT LAST LINE DOES CONFLICT!! Run this ONLY: sudo dpkg -i debs/libglfw3*deb; sudo apt-get install -f. Otherwise, you will have the same problem as above, when it uninstalls the window manager.
  • (still unstable) if you need CUDA for some reason, for example if you are using any deep learning techniques, follow the instructions here first before you do the Kinect2 installation, and then proceed with the Kinect2 installation instructions continuing to ignore their advice on CUDA
    • I would caveat this caveat by saying: If you really need ROS and CUDA and you need to guarantee that they work, your safest option is to install CUDA first according to the NVIDIA guides (the above guide should work too), and then use ROS on Docker. Remember that ROS is essentially a middleware communication framework, nothing more, while CUDA is hardware drivers and development packages, so CUDA is the thing that your PC needs to actually be on it. Any other potentially PC-breaking libraries are also best used as Docker containers, as breaking your Docker environment has virtually (pun intended) zero consequences.
  1. Install missing packages.
  • sudo apt-get install ros-indigo-visualization-msgs
  1. Update PCL
  • There is an issue with PCL and Boost being compiled under different c++ standards. To rebuild PCL from source:
    • cd ~/Software
    • git clone https://github.com/PointCloudLibrary/pcl.git
    • cd pcl && mkdir build && cd build
    • cmake ..
    • make -j2
    • sudo make -j2 install

  1. Clone and link the perception packages to your workspace:

    • cd ~/Software/HLP-R
    • git clone --recursive https://github.com/HLP-R/hlpr_perception.git
    • ln -s ~/Software/HLP-R/hlpr_perception/ ~/vector_ws/src/

  2. Build your workspace.

    • cd ~/vector_ws
    • catkin_make

  3. Source your catkin workspace (or close and reopen the terminal if it is in your .bashrc)

    • source ~/vector_ws/devel/setup.bash

  4. Launch the segmentation node.

    • roslaunch hlpr_segmentation pc_seg.launch OR roslaunch hlpr_nonplanar_segmentation nonplanar_seg.launch OR roslaunch hlpr_single_plane_segmentation hello_world.launch (see below for how these packages differ from each other)

  5. In another window, launch the feature extraction node.

    • roslaunch hlpr_feature_extraction viz.launch (This file also launch segmentation nodes. So close the previously launched segmentation node) OR roslaunch hlpr_nonplanar_feature_extraction nonplanar_ft_ex.launch (This one needs a segmentation node to be run separately.) (see below for how these packages differ from each other)

NOTE - Make sure you launch the correct feature extraction node for the respective segmentation nodes. As of now, they do not work interchangeably. Look in the sections below for more information.

NOTE - If you run into Eigen errors while launching the segmentation nodes, follow the example here and insert Eigen::DontAlign at the appropriate places in the source code.

Perception Pipeline

Description of Packages

  • hlpr_perception_msgs defines the set of ROS messages to be used for communication along the perception pipeline so that modular packages can be used interchangeably.
  • hlpr_segmentation is a segmentation node that uses PCL MultiplaneSegmentation for plane segmentation and output segmented clusters. It is optimized for fast processing but may not pick up short/small objects on a plane.
  • hlpr_nonplanar_segmentation is a segmentation node that can do nonplanar segmentation. It can be used to segment objects in a scene that does not necessarily contain a tabletop. It is also fast but not good for small objects.
  • hlpr_single_plane_segmentation is a segmentation node that uses RANSAC instead of MultiplaneSegmentation for plane segmentation. It is better at picking up short/small objects on the plane but trades off the efficiency from MultiplaneSegmentation. Feature extraction has not been implemented for this segmentation yet.
  • hlpr_feature_extraction takes segmented clusters and outputs features extracted for each of the clusters. It currently ONLY works with clusters published by hlpr_segmentation.
  • hlpr_nonplanar_feature_extraction works with hlpr_nonplanar_segmentation and outputs corresponding features extracted from received clusters.
  • hlpr_object_labeling and hlpr_knowledge_retrieval are example packages for recording object features, labelling objects in a scene as well as retrieving knowledge given a labelled object. Inspect the launch files for details about how to use these packages.

Parameters

  • You can set any parameters using the rosparam convention in your launch file. For an example, see launch/hello_world.launch.
  • You can set any of the following parameters:
    • dt: distance threshold for segmentation (float)
    • ct: color threshold for segmentation (float) (currently set a very high (e.g. 100000) threshold to do proximity only segmentation)
    • v: set hue value to select the cluster of interest (float)
    • m: set whether to do merging or not (boolean)
    • t: set hue threshold for merging
    • z: set depth threshold for merging (float)
    • p: set whether to pre process the point cloud or not (boolean)
    • c: set color segmentation options, 0,1,2 correspond to none, rgb, hue
    • src: point cloud source options 0,1,2 correspond to ROS, OPENNI, and KinectV2 respectively (int).
    • rt: name of the ros topic of the incoming point cloud (string)
    • out: output options 0,1,2 corresponding to none, IRCP, ROS (int). IRCP support may not exist, exercise caution.
    • comm: Communication options 0,1,2 corresponding to none, IRCP, ROS (int). IRCP support may not exist, exercise caution.
    • b: set whether to display all bounding boxes or just the selected one (boolean)
    • sh: set saturation hack on/off (boolean)
    • st: set saturation threshold for the hack (float)
    • sv: set saturation value to be mapped to (float)
    • ri: set robot id for ircp (byte)
    • pr: set the freenect2 processor options 0,1,2 correspond to CPU, OPENCL, and OPENGL respectively (int)
    • fn: filter noise, warning slows things down (bool)
    • nv: no vizualization (bool)
Clone this wiki locally