MaskPlanner: a Framework for 3D Learning-Based Object-Centric Motion Generation and Applications to Robotic Spray Painting

Preprint / Website / Dataset

Gabriele Tiboni, Raffaello Camoriano, Tatiana Tommasi

IEEE Transactions on Robotics (T-RO), 2026

Installation

This repository is designed for flexible experimentation. Clone the repository and install all dependencies:

1. Dataset

   1.1 Download Extended PaintNet Dataset from zenodo.org/records/14967945

   1.2 export PAINTNET_ROOT=<path/to/dataset/>

2. Clone repo and install basic dependencies:

   git clone <this-repo>
   cd MaskPlanner
   pip install -r requirements.txt

3. Set up CUDA and PyTorch:

   3.1 Set up a full CUDA toolkit installation (not just the pytorch runtime version), i.e. the command nvcc --version must be working.

   3.2 Install pytorch according to your CUDA version: https://pytorch.org/get-started/locally/

   3.3 Install pytorch3d: https://github.com/facebookresearch/pytorch3d/blob/main/INSTALL.md

   3.4. Pay attention to the compatibility among the above packages according to the installed versions. See below for the specific versions this code has been tested on.

4. (optional) Set up a custom working directory for the training runs and wandb login:

   export WORKDIR=<root/to/runs/dir>
   
   # login to wandb if you want to log runs on wandb with: python [...] wandb=online

Versioning

This code has been tested on:

• python=3.7, nvcc=10.2, torch=1.9.0, pytorch3d=0.7.0
  • pytorch3d=0.7.0 was installed from source from Github: pip install "git+https://github.com/facebookresearch/pytorch3d.git@v0.7.0"
• python=3.8, nvcc=11.6, torch=1.13.1, pytorch3d=0.7.2
  • In this case, pytorch3d 0.7.2 could only be installed through the Anaconda cloud (conda install pytorch3d -c pytorch3d, or through the specific label pytorch3d-0.7.2-py38_cu116_pyt1131.tar.bz2)
• python=3.8, nvcc=12.1, torch=2.2.0, pytorch3d commit c292c71c1adb0712c12cf4fa67a7a84ad9b44e5c
  • pytorch3d installed from main branch on Github through command: pip install "git+https://github.com/facebookresearch/pytorch3d.git" (HEAD commit at the time of writing is: c292c71c1adb0712c12cf4fa67a7a84ad9b44e5c)

Getting started

Training MaskPlanner

1. Quick training for debugging (Windows category)

   # Implicit, using `maskplanner` as an alias for multiple config files
   python train_maskplanner.py config=[maskplanner,windows_v2,longx_v2,debug] seed=42
   
   # Explicit, using the full list of config file names and some parameters explicitly. Equivalent to the above command.
   python train_maskplanner.py config=[asymm_chamfer_v9,delayMasksLoss,traj_sampling_v2,sched_v9,windows_v2,longx_v2] \
                               wandb=disabled  \
                               epochs=20 \
                               eval_freq=10 \
                               batch_size=2 \
                               debug=true \
                               no_save=true \
                               seed=42

2. Complete training (Windows category)

   python train_maskplanner.py config=[maskplanner,windows_v2,longx_v2] seed=42

Config files defined in the config=[...] parameters can be found at configs/maskplanner. If a parameter appears in multiple config files, the order of the config files matter (later configs overwrite preceeding configs). A parameter may always be explicitly specified on the command line, talking ultimate priority (e.g. seed=42). In the examples above, the config name maskplanner is an alias that includes multiple config files at the same time (i.e. asymm_chamfer_v9,delayMasksLoss,traj_sampling_v2,sched_v9). See aliases in utils/args.py:config_aliases. See more examples at the top of train_maskplanner.py.

Reproduce paper results

python train_maskplanner.py config=[maskplanner,cuboids_v2,longx_v2] seed=42
python train_maskplanner.py config=[maskplanner,windows_v2,longx_v2] seed=42
python train_maskplanner.py config=[maskplanner,shelves_v2,longx_v2] seed=42
python train_maskplanner.py config=[maskplanner,containers_v2,longx_v2] seed=42

Citation

If you find this repository useful, please consider citing:

@misc{tiboni2025maskplanner,
      title={MaskPlanner: Learning-Based Object-Centric Motion Generation from 3D Point Clouds}, 
      author={Gabriele Tiboni and Raffaello Camoriano and Tatiana Tommasi},
      year={2025},
      eprint={2502.18745},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2502.18745}, 
}

Acknowledgments

We acknowledge the EFORT group for providing object mesh files, expert trajectory data, access to a proprietary spray painting simulator and to specialized painting robot hardware for the real-world experimental evaluation.