Skip to content
Python and ROS (C++) utilities for the ContactDB dataset
Python C++ CMake Shell
Branch: master
Clone or download
Latest commit 53661cb Jul 11, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
calibrations First commit of all code. Need to clean up Apr 21, 2019
data documentation Apr 22, 2019
docs Update deps.md Jun 29, 2019
include/deepgrasp_utils First commit of all code. Need to clean up Apr 21, 2019
launch Update thermal.launch Jun 23, 2019
rviz_files
scripts updated a lot of names Apr 27, 2019
src First commit of all code. Need to clean up Apr 21, 2019
.gitignore First commit of all code. Need to clean up Apr 21, 2019
CMakeLists.txt update project name to match package name Jul 4, 2019
LICENSE Initial commit Apr 5, 2019
README.md Update README.md Jul 12, 2019
contactmap_example.gif Add files via upload May 8, 2019
package.xml updated a lot of names Apr 27, 2019
requirements.txt documentation Apr 22, 2019

README.md

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

This repository contains code to create the human grasp contact maps, presented in the paper

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging - Samarth Brahmbhatt, Cusuh Ham, Charles C. Kemp, and James Hays, CVPR 2019

Paper (CVPR 2019 Oral) | Explore the dataset | Poster | Slides

Please see contactdb_prediction for code to perform the contactmap prediction experiments presented in the paper.

This code is in the process of being documented. Feel free to open an issue if you need urgent help.

Dataset Download:

We offer both processed and raw forms of the data.

Processed Data

Contact Maps (Textured Meshes) (11.5 GB). If you also need the RGBD-Thermal images, 6-DOF object poses, and image masks, use this Dropbox link (91 GB) instead.

Raw Data

ROS bagfiles (1.46 TB): Compressed 30 Hz RGB-D-Thermal data streams. You will also need the Object 3D Models (180 MB) to create contact maps from this raw data.

3D Printing

This spreadsheet has links to 3D models suitable for printing, along with scaling information. We printed the objects at 15% hexagonal honeycomb infill with PLA material. Contact Samarth Brahmbhatt at samarth.robo@gatech.edu if links in that spreadsheet are dead.

We designed a mount for rigidly attaching the FLIR Boson camera to the Kinect v2, you can find it here.

Note: these models are suitable for 3D printing, but some characteristics like sharp triangles make them unsuitable for texture mapping. Please use the re-meshed .ply models from the Processed Data section above for texture mapping. All the object poses, camera poses etc. are defined with respect to those re-meshed models.

This article is a good resource for getting files ready for 3D printing. Netfabb has a free student license.

Setup

Using Processed Data

The repository includes some handy Python scripts in the scripts directory. You can ignore all the C++ code.

  1. Install Open3D, numpy, and matplotlib.
  2. Download this repository: git clone https://github.com/samarth-robo/contactdb_utils.
  3. Download the Processed Data, and make the symlink: ln -s DOWNLOAD-DIR data/contactdb_data.
  4. If you want to perform machine learning experiments, check out the contactdb_prediction repository.

Using Raw Data

Our code is a ROS package that has been tested on Ubuntu 16.04 LTS with ROS Kinetic Kame.

  1. Install dependencies
  2. Set up a Catkin workspace in ~/catkin_ws.
  3. Download this repository and my fork of cv_camera and compile the ROS package:
$ cd ~/catkin_ws/src
$ git clone https://github.com/samarth-robo/contactdb_utils
$ git clone https://github.com/samarth-robo/cv_camera
$ cd ..
$ catkin_make
  1. Download Raw Data and 3D Models, and make symlinks: ln -s DATA-DOWNLOAD-DIR data/contactdb_data, ln -s 3D-MODEL-DOWNLOAD-DIR data/contactdb_3d_models.
  2. Download and setup the ICP code for object pose estimation:
$ git clone https://github.com/samarth-robo/ICP-gui
$ cd ICP-gui
$ mkdir build && cd build && make -j
  1. Create a symlink to the build directory:
$ cd contactdb_utils
$ ln -s <ICP-gui dir>/build ICP_build

Visualizing Contact Maps

This script applies our post-processing to the contact map, and shows it in interactive 3D. You need to either download the processed data, or process the raw data before running it.

cd contactdb_utils/scripts
python show_contactmap.py --object_name <object name> --session <participant number 1-50> --instruction <use | handoff>

Documents:

Citation

@inproceedings{brahmbhatt2018contactdb,
  title={{ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging}},
  author={Samarth Brahmbhatt and Cusuh Ham and Charles C. Kemp and James Hays},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2019},
  note={\url{https://contactdb.cc.gatech.edu}}
}
You can’t perform that action at this time.