WEAKLY SUPERVISED POINT CLOUD UPSAMPLING VIA OPTIMAL TRANSPORT
(Official TensorFlow Implemention, the code is modified from PU-GAN)
Existing learning-based methods usually train a point cloud upsampling model with synthesized, paired sparse-dense point clouds. However, the distribution gap between synthe- sized and real data limits the performance and generalization. To solve this problem, we innovatively regard the upsamplig task as an optimal transport (OT) problem from sparse to dense point cloud. Further we propose PU-CycGAN, a cycle network that consists of a Densifier, Sparsifier and two discriminators. It can be directly trained for upsampling with unpaired real sparse point clouds, so that the distribution gap can be filled via the learning. Especially, quadratic Wasserstein distance is introduced for the stable training.
-
Create conda virtual environment
# The recommended Python version is 3.6 conda create --name pucycgan python=3.6 conda activate pucycgan # Install TensorFlow conda install tensorflow-gpu=1.13.1 # Some warnings can be avoided by using numpy < 1.17 conda install numpy=1.16.6
-
Compiling TF operators
- Check the
tf_XXX_compile.sh
script in each sub file under thetf_ops
directory, and modify the location of the Python interpreter, CUDA runtime API and TensorFlow in the shell script to correspond to the local machine. - Execute each
tf_xxx_compile.sh
successively, or copycompile.sh
fromdocker
directory totf_ops
directory for execution; Then execute thecompile_render_balls_so.sh
script
- Check the
-
Clone the repository:
git clone https://github.com/cognaclee/PU-CycGAN cd PU-CycGAN
-
Training
Download PU-GAN or our training dataset we collected from KITTI and Semantic3d (we also organize the data into HDF5 files, and the keys of the dataset is
Kitti
andsemantic3d
) from Google Drive and then put it in thedata
folder.python pu_gan.py --phase=train
-
Testing
Download PU-GAN or our testing dataset from Google Drive and put it into the
data
folder.python pu_gan.py --phase=test --data_dir=path/to/testing/dataset --log_dir=path/to/trained/model --out_folder=path/to/output/folder
-
Evaluation
# calculate the uniform metric cd evaluation_code cmake . make # The second argument is the mesh, and the third one is the predicted points ./evaluation Icosahedron.off Icosahedron.xyz python evaluate.py --gt=path/to/groundtruth --pred=/path/to/prediction
The results are saved in
evaluation.csv
in the folder where the prediction file is located.
@inproceedings{li2022weakly,
title={Weakly Supervised Point Cloud Upsampling VIA Optimal Transport},
author={Li, Zezeng and Wang, Weimin and Lei, Na and Wang, Rui},
booktitle={ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={2564--2568},
year={2022},
organization={IEEE}
}