Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
layout
default

David Watkins-Valls, Jacob Varley, and Peter Allen

Columbia Robotics Lab


Abstract

This work provides an architecture that incorporates depth and tactile information to create rich and accurate 3D models useful for robotic manipulation tasks. This is accomplished through the use of a 3D convolutional neural network (CNN). Offline, the network is provided with both depth and tactile information and trained to predict the object's geometry, thus filling in regions of occlusion. At runtime, the network is provided a partial view of an object and tactile information is acquired to augment the captured depth information. The network can then reason about the object's geometry by utilizing both the collected tactile and depth information. We demonstrate that even small amounts of additional tactile information can be incredibly helpful in reasoning about object geometry. This is particularly true when information from depth alone fails to produce an accurate geometric prediction. Our method is benchmarked against and outperforms other visual-tactile approaches to general geometric reasoning. We also provide experimental results comparing grasping success with our method.

Training Data Example


Completion Examples

We created a database of training examples and completions of those examples. See completions [here]({{ site.url }}/completions_view/).


Video

<iframe width="560" height="315" src="https://www.youtube.com/embed/VYACdyN3Bg0" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

Downloads

Source code + Trained Model (Keras 2.0)

Training data

  • 726 pointclouds for each of 608 objects, as well as the 428,340 corresponding 403 aligned x,y training example pairs from the Grasp and YCB Databases. here

Ground Truth Meshes

The ground truth meshes are not ours to distribute. To get them, please register at:


Citation

This work was accept by ICRA 2019. Arxiv link here

About

No description, website, or topics provided.

Resources

License

Releases

No releases published

Packages

No packages published