Skip to content


Folders and files

Last commit message
Last commit date

Latest commit



9 Commits

Repository files navigation

Rethinking Pseudo-LiDAR Representation


This project is based on our ECCV 2020 paper. In this work, we perform an in-depth investigation and observe that the efficacy of pseudo-LiDAR representation comes from the coordinate transformation, instead of data representation itself. Based on this observation, we design an image based CNN detector named PatchNet, which is more generalized and can be instantiated as pseudo-LiDAR based 3D detectors. Moreover, the pseudo-LiDAR data in our PatchNet is organized as the image representation, which means existing 2D CNN designs can be easily utilized for extracting deep features from input data and boosting 3D detection performance.


Besides, this project also provides the pytorch implenmentation of FPointNet and pseudo-LiDAR with competitive accuracies.


If you find our work useful in your research, please consider citing:

author = {Ma, Xinzhu and Liu, Shinan and Xia, Zhiyi and Zhang, Hongwen and Zeng, Xingyu and Ouyang, Wanli},
title = {Rethinking Pseudo-LiDAR Representation},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2020}}



Download this repository (tested under python3.6, pytorch1.1 and ubuntu 14.04). There are also some dependencies like cv2, yaml, tqdm, etc., and please install them accordingly:

cd #root
pip install -r requirements

Then, you need to compile the evaluation script:

cd root/tools/kitti_eval

Prepare your data

First, you should download the KITTI dataset, and organize the data as follows (*: this is an empty directory to store the data generated in subsequent steps):

      |pickle_files *
          |depth *
          |pseudo_lidar (optional for Pseudo-LiDAR)*
          |velodyne (optional for FPointNet)
          |depth *
          |pseudo_lidar (optional for Pseudo-LiDAR)*
          |velodyne (optional for FPointNet)

Then you need to prepare your depth maps and put them to data/KITTI/object/training/depth. To provide ease of use, we also provide the estimated depth maps (these data generated from the pretrained models provided by DORN and Pseudo-LiDAR).

Monocular (DORN) Stereo (PSMNet)
trainval(~1.6G), test(~1.6G) trainval(~2.5G)

Finally, generate the training data using provided scripts :

cd #root/tools/data_prepare
python --gen_train --gen_val --gen_val_rgb_detection --car_only
mv *.pickle ../../data/KITTI/pickle_files

You can also generate the data for pseudo-LiDAR or FPointNet models using Note you need to run before generating the data for pseudo-LiDAR based models.


Move to the patchnet workplace (we also provide the codes of FPointNet and pseudo-LiDAR, and welcome to try them) and train the mode (also need to modify the path of pickle files in config file):

 cd #root
 cd experiments/patchnet
 python ../../tools/ --config config_patchnet.yaml

For ease of use, we also provide the pretrained models, and the performances (AP3D@11 for the pretained models / original paper) are shown as follows:

Models AP3D11@easy AP3D11@mod. AP3D11@hard
PatchNet@monocular 34.8 / 35.1 22.0 / 22.0 19.5 / 19.6
PatchNet@stereo 67.4 / 65.9 42.3 / 42.5 38.4 / 38.5

Note that the pre-trained models are trained in a shorter training schedule (100 epoches).


Generate the results using the trained model:

 python ../../tools/ --config config_patchnet.yaml --e

and evalute the generated results using:

../../tools/kitti_eval/evaluate_object_3d_offline_ap11 ../../data/KITTI/object/training/label_2 ./output


../../tools/kitti_eval/evaluate_object_3d_offline_ap40 ../../data/KITTI/object/training/label_2 ./output

Note: all the above commands are packed in the, and you can use following command for both training and testing:


Change Logs

  • 08/17/2020: add a script for visualization


This code benefits from the excellent work FPointNet and use the pre-trained models provided by DORN and pseudo-LIDAR. Please also consider citing these works.


If you have any question about this project, please feel free to contact


Code release for "Rethinking Pseudo-LiDAR Representation (ECCV2020)".







No releases published


No packages published