PatchScene: Patch-based Voxel Diffusion for Large-Scale LiDAR Scene Completion

Paper | Supplementary | Project Page | Code | Checkpoint |

Qingdong Xu^1,*, Jiajun Zhu^1,2,*, Shilin Zhu^4,*, Xinjing He⁵, Chao Lu², Huanran Wang², Jiyao Zhang^3,†

¹MEGVII Technology    ²Qianli Technology    ³Peking University    ⁴Northeastern University, China    ⁵Northwestern Polytechnical University

^*Equal Contribution    ^†Corresponding Author

---

Abstract

We propose PatchScene, a novel diffusion-based framework for large-scale LiDAR scene completion. Unlike existing methods that rely on global latent representations or dense voxel grids, PatchScene adopts a patch-based voxel diffusion paradigm that explicitly generates fine-grained geometry within localized 3D regions. To ensure coherent reconstruction at both spatial and temporal scales, we introduce a confidence-guided spatio-temporal fusion mechanism that integrates overlapping patches and adjacent frames in a unified generative process. Furthermore, we design an Annular-Flow diffusion strategy that leverages the radial density pattern of LiDAR scans to progressively propagate high-fidelity information from near-range to far-range regions, enabling spatially unbounded scene completion. Extensive experiments on the SemanticKITTI benchmark demonstrate that PatchScene achieves state-of-the-art performance across all standard metrics, surpassing previous approaches in both geometric accuracy and temporal consistency. Remarkably, the model trained on 20 m LiDAR ranges generalizes effectively to 50 m scenes without retraining, highlighting its strong scalability and generalization capability for real-world autonomous driving applications.

---

News

• [2026/06] PatchScene has been accepted to CVPR 2026.
• [2026/06] Project page released.
• [2026/06] Code and pretrained models released.

---

Installation

conda create -n patchscene python=3.9 pip=23.3.1 -y
pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu121

pip install numba==0.58.1 numpy==1.24.3 setuptools==75.1.0

pip install scikit-image==0.18.2 scikit-learn==0.24.2
pip install tqdm fire einops pyrender pyrr trimesh rich
pip install --no-deps pytorch-lightning==1.6.1

pip install "PyYAML>=5.4" "packaging>=17.0" "pyDeprecate==0.3.2"
pip install torchmetrics==1.5.2
pip install tensorboard==2.14.0 

pip install timm==1.0.15
pip install natsort ipdb

pip install open3d==0.19.0

---

Data Preparation

SemanticKITTI

1. Download the SemanticKITTI dataset.

2. Generate dense ground-truth maps (accumulated full point clouds in ego coordinates) for each sequence. Follow the data preparation procedure provided in LiDiff.

3. Generate annotation pickles:

python tools/prepare_data.py --data_root /path/to/semanticKITTI

This creates semantic_kitti_infos_new_train.pkl and semantic_kitti_infos_new_val.pkl.

Expected directory structure:

/path/to/semanticKITTI/
├── semantic_kitti_infos_new_train.pkl
├── semantic_kitti_infos_new_val.pkl
└── dataset/
    └── sequences/
        ├── 00/
        │   ├── velodyne/         # raw LiDAR scans (.bin)
        │   ├── labels/           # semantic labels (.label)
        │   ├── calib.txt
        │   ├── poses.txt
        │   └── map_clean.npy     # precomputed dense map (ego coordinates)
        ├── ...
        └── 10/

Note: The dataset split follows the standard SemanticKITTI convention: sequences 00–07, 09, 10 for training; sequence 08 for validation.

KITTI-360 (optional)

For KITTI-360 support, use configs/train_kitti360_config.py and test_kitti360.py analogously.

---

Pre-trained Model

Checkpoint	Description	Link
epoch=99.ckpt	SemanticKITTI checkpoint (trained with 8× NVIDIA L20 48 GB GPUs).	Download

Place the checkpoint under results/ and update the model_path variable in test_semantickitti.py to point to your checkpoint.

---

Usage

Training

Single GPU:

python train.py \
  --config_file configs/train_semantickitti_config.py \
  --batch_size 14 \
  --num_workers 18 \
  --save_every_epoch 40 \
  --dataset_name "semanticKITTI" \
  --use_lidar_cond True \
  --use_rgb_cond False \
  --use_position True \
  --lr 4e-4

Multi-GPU (DDP): PyTorch Lightning automatically uses all visible GPUs when devices=-1:

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python train.py \
  --config_file configs/train_semantickitti_config.py \
  --batch_size 14 \
  --num_workers 18

Inference

SemanticKITTI:

python test_semantickitti.py \
  --generate_method Ring_Spread_V2 \
  --steps 15 \
  --start_index 0

KITTI-360:

python test_kitti360.py \
  --generate_method Ring_Spread_V2 \
  --steps 15 \
  --start_index 0

---

Citation

If you find our work useful, please consider citing:

@inproceedings{xu2026patchscene,
  title={PatchScene: Patch-based Voxel Diffusion for Large-Scale LiDAR Scene Completion},
  author={Xu, Qingdong and Zhu, Jiajun and Zhu, Shilin and He, Xinjing and Lu, Chao and Wang, Huanran and Zhang, Jiyao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2026}
}

---

License

This project is released under the MIT License. See LICENSE for details.

Acknowledgements

This codebase builds upon the LAS-Diffusion framework for the base diffusion model architecture. We thank the authors for their excellent work.