Skip to content

Repository gathering all code related to the paper GRASPA, Bottarel, Vezzani, Pattacini Natale, IEEE RA-L, vol. 5, no. 2, pp. 836-843, April 2020.)

License

MIT, CC-BY-4.0 licenses found

Licenses found

MIT
LICENSE.txt
CC-BY-4.0
LICENSE_YCB.txt
Notifications You must be signed in to change notification settings

hsp-iit/GRASPA-benchmark

Repository files navigation

Welcome to GRASPA!

GitHub release (latest by date) GitHub Workflow Status paper-ieeexplore paper-arxiv gitpod

GRASPA (GRASPA is a Robot Arm graSping Performance benchmArk) is a benchmarking protocol and a set of metrics to evaluate the performance of grasp planners. It aims to fairly compare methodologies tested on different robots by measuring and accounting for platform limitations that might affect the performance of the algorithm itself.

Version 1.0

Version 1.0 features:

  • 3 printable layouts of predefined grasping scenarios (composed of a subset of YCB objects and localization markers) to enhance test reproducibility.
  • A protocol to assess the robot reachability and the calibration of the vision system within the defined grasping setup area.
  • Usage of a popular grasp quality metric to evaluate user-provided grasping poses.
  • A score to assess the grasp stability during the practical execution on the robot.
  • The possibility to benchmark the algorithm either in isolation or in the clutter, with the definition of a further metric to evaluate the obstacle avoidance in the clutter modality.
  • A composite score to quantify the overall performance of the algorithm accounting for reachability and calibration limits of the test platform.

Preview

Overview

How to install

Using GitPod

We have packaged a GitPod environment with every dependency compiled and already installed. Click on the button to open the cloud environment in your browser.

Gitpod

Locally

To ease the installation process, please use the Docker image we prepared.

docker pull ghcr.io/hsp-iit/graspa-benchmark:latest

How to run

Using GitPod

GitPod is a cloud-based development environment that allows users to test code from a GitHub repository without having to set up the environment and dependencies on their local machine. We have set up such an environment for this repo to allow users to test the code in the most hassle-free way.

To further smoothen user experience, our GitPod instance comes with a Fluxbox desktop environment packed in. Once the environment is ready (it takes a while), GitPod will signal the user that a service is available on port 8080. By clicking on the Open Browser option, you will be automatically connected to the in-browser desktop. Otherwise, click Remote Explore tab in Gitpod, the Gitpod Workspace dropdown shows several ports. Select Open Browser for the 8080 port and click on vnc_auto.html to visualize the remote desktop.

To test the code on the data that we gathered for the paper and see the outcome, simply open a terminal in the remote desktop and type

    cd /path/to/GRASPA-benchmark
    $ ./src/script_launcher.sh

and the bash script will perform every step required to compute the benchmark score of our platform.

Locally

You can use the Docker image you just pulled. Fire it up with

docker run -it --rm -p 8080:8080 ghcr.io/hsp-iit/graspa-benchmark

and then navigate from your favourite web browser to http://localhost:8080. At this point, open a terminal and proceed as outlined in the GitPod instructions section.

Once the data required by the benchmark are available (see here instructions on how to collect them), all the scores can be computed by:

  1. Properly pointing to the folders including the data in the src/script_launcher.sh. More information on how to fill correctly the script is available here.
  2. Launching the script:
    cd $GRASPA_DIR
    $ ./src/script_launcher.sh
    
    An example of the benchmark output is available here.

Resources&Tools

GRASPA 1.0 comes with several resources and tools:

[1]This repo includes a subset of the YCB object dataset meshes for user convenience. We re-distribute these files under the same Creative Commons Attribution 4.0 International (CC BY 4.0) license.

About

Repository gathering all code related to the paper GRASPA, Bottarel, Vezzani, Pattacini Natale, IEEE RA-L, vol. 5, no. 2, pp. 836-843, April 2020.)

Resources

License

MIT, CC-BY-4.0 licenses found

Licenses found

MIT
LICENSE.txt
CC-BY-4.0
LICENSE_YCB.txt

Stars

Watchers

Forks

Contributors 4

  •  
  •  
  •  
  •