MaskUKF

An Instance Segmentation Aided Unscented Kalman Filter for 6D Object Pose and Velocity Tracking

Frontiers in Robotics and AI

Paper | Video

Reproducing the experiments

We support running the experiments via the provided Docker image.

If you want to install the repository manually, please refer to the recipe contained in the Dockerfile.

1. Pull the docker image:

   docker pull ghcr.io/hsp-iit/mask-ukf:latest

2. Launch the container:

   docker run -it --rm --user user --env="DISPLAY" --net=host ghcr.io/hsp-iit/mask-ukf:latest

3. Clone and build the project:

   git clone https://github.com/hsp-iit/mask-ukf.git
   cd mask-ukf
   mkdir build && cd build
   cmake ../
   make

4. Download and unzip the accompanying data and the YCB-Video model set:

   cd /home/user/mask-ukf
   bash misc/download_accompanying_data.sh
   bash misc/download_ycb_models.sh

5. Run the experiments (optional):

   cd /home/user/mask-ukf
   bash test/test_all.sh

   The accompanying data contains the pre-evaluated results. If desired, the results can be re-evaluated using the above command.

6. Run the evaluation:

   cd /home/user/mask-ukf
   bash evaluation/evaluate_<mask_set>_<metric>_<algorithm>.sh

   <mask_set> can be mrcnn (Mask R-CNN) or posecnn (PoseCNN), <metric> can be add_s (ADD-S) or rmse (RMSE) and <algorithm> can be empty (MaskUKF), icp (ICP) or densefusion (DenseFusion).

7. Visualize the results:

   cd /home/user/mask-ukf
   python3 evaluation/renderer/renderer.py --algorithm <algorithm> --mask_set <mask_set> --object <object_name> --video_id <video_id>

   <algorithm> can be mask-ukf (MaskUKF), icp (ICP) or dense_fusion (DenseFusion), <mask_set> is as above, <object_name> is e.g. 002_master_chef_can and <video_id> is the YCB-Video video id, e.g. 0048.

In order to run the visualizer it could be required to temporarily execute xhost + in a console outside of Docker in order to allow the container accessing the X server facilities. The command can be run even after the container has been already launched.

Citing MaskUKF

If you find the MaskUKF code useful, please consider citing the associated publication:

@ARTICLE{10.3389/frobt.2021.594583,
AUTHOR={Piga, Nicola A. and Bottarel, Fabrizio and Fantacci, Claudio and Vezzani, Giulia and Pattacini, Ugo and Natale, Lorenzo},
TITLE={MaskUKF: An Instance Segmentation Aided Unscented Kalman Filter for 6D Object Pose and Velocity Tracking},
JOURNAL={Frontiers in Robotics and AI},
VOLUME={8},
PAGES={38},
YEAR={2021},
URL={https://www.frontiersin.org/article/10.3389/frobt.2021.594583},
DOI={10.3389/frobt.2021.594583},
ISSN={2296-9144}
}

and/or the repository itself by pressing on the Cite this respository button in the About section.

Maintainer

This repository is maintained by:

	@xenvre