This pose estimation method combines the PointNet model with the voting and clustering process described in Rotational Subgroup Voting and Pose Clustering for Robust 3D Object Recognition.
This project was part of my master thesis, for a deeper explanation of the pose estimation process, the report can be found here.
Run the configuration script in order to create a virtual environment for the repository, install all the necessary packages and create the necessary folders.
cd ./config
chmod +x config.sh
./config.shAdd the repository path to the PYTHONPATH environment variable. Edit your .profile and add the following lines:
gedit ~/.profile# set PATH for the RSVnet repository
export RSV_REPOSITORY_PATH=/path/to/the/repository
if [ -d "$RSV_REPOSITORY_PATH/" ]; then
export PYTHONPATH=$PYTHONPATH:"$RSV_REPOSITORY_PATH/Linemod/object_classification"
export PYTHONPATH=$PYTHONPATH:"$RSV_REPOSITORY_PATH/config"
export PYTHONPATH=$PYTHONPATH:"$RSV_REPOSITORY_PATH/utilities"
export PYTHONPATH=$PYTHONPATH:"$RSV_REPOSITORY_PATH/PointNet"
export PYTHONPATH=$PYTHONPATH:"$RSV_REPOSITORY_PATH/RSV"
export PYTHONPATH=$PYTHONPATH:"$RSV_REPOSITORY_PATH/RSVnet"
fiSource the .profile to update the changes.
source ~/.profileThis project was build using Ubuntu 18.04 and also requires the installation of CUDA in order to use tensorflow-gpu. I followed these tutorial in order to install it:
Make sure that the paths defined in config/configuration.py are all correct.
The first step is to add the Linemod data to the correct folder. The pointclouds should be in .ply format and the object of interest pose in a .txt file. The pointclouds should be placed in the Linemod/ply whereas the pose files go to Linemod/pose.
Once the data is added, the following step consist on generating the dataset for training the different PointNet models and the regression model. To do so, we need to run:
python Linemod/object_classification/main.py
python Linemod/RSVnet/main.py As a result each script generates a set of .h5 files with the data necessary for training and evaluating the models. The data can be found in:
- Linemod/object_classification/data
- Linemod/RSVnet/data
Regarding the data for the scene classification model, it can be downloaded from here. The process for obtaining the dataset is not implemented in this project. However the procedure is explained in detail in the master thesis report.
The dataset files can be split in different groups for training, test and evaluation. Each model has its own data folder:
- RSVnet/data
- PointNet/object_classification/data
- PointNet/scene_classification/data
In order to train the classification models, we can run:
python PointNet/scene_classification/train.py
python PointNet/object_classification/train.py The trained model will be saved into:
- PointNet/scene_classification/models
- PointNet/object_classification/models
In order to evaluate the classification models, we can run:
python PointNet/scene_classification/test.py
python PointNet/object_classification/test.py The regression model can be trained by running:
python RSVnet/train.py And evaluated with:
python RSVnet/test.py Once all the models have been trained, the main pose estimation process can be run.
python src/main.py This project has the following main python dependencies:
- Rotational Subgroup Voting and Pose Clustering for Robust 3D Object Recognition by Anders Glent Buch, Lilita Kiforenko, Dirk Kraft.
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation by Qi et al. (CVPR 2017 Oral Presentation). Code and data released in GitHub
- PointNe-Keras: Keras implementation for Pointnet by garyli1019 .