GauHuman: Articulated Gaussian Splatting from Monocular Human Videos

Shoukang Hu; Ziwei Liu

S-Lab, Nanyang Technological University

CVPR 2024

GauHuman learns articulated Gaussian Splatting from monocular videos with both fast training (1~2 minutes) and real-time rendering (up to 189 FPS).

📖 For more visual results, go checkout our project page

This repository will contain the official implementation of GauHuman: Articulated Gaussian Splatting from Monocular Human Videos.

📣 Updates

[12/2023] Training and inference codes for ZJU-Mocap_refine and MonoCap are released.

🖥️ Requirements

NVIDIA GPUs are required for this project. We recommend using anaconda to manage the python environments.

    conda create --name gauhuman python=3.8
    conda activate gauhuman
    conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.8 -c pytorch -c nvidia
    pip install submodules/diff-gaussian-rasterization
    pip install submodules/simple-knn
    pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl
    pip install -r requirement.txt

Tips: We implement the alpha mask loss version based on the official diff-gaussian-rasterization.

Set up Dataset

Please follow instructions of Instant-NVR to download ZJU-Mocap-Refine and MonoCap dataset.

Download SMPL Models

Register and download SMPL models here. Put the downloaded models in the folder smpl_models. Only the neutral one is needed. The folder structure should look like

./
├── ...
└── assets/
    ├── SMPL_NEUTRAL.pkl

🚋 Training

Training command on ZJU_MoCap_refine dataset

bash run_zju_mocap_refine.sh

Training command on MonoCap dataset

bash run_monocap.sh

🏃‍♀️ Evaluation

Evaluation command on ZJU_MoCap_refine dataset

bash eval_zju_mocap_refine.sh

Evaluation command on MonoCap dataset

bash eval_monocap.sh

🤟 Citation

If you find the codes of this work or the associated ReSynth dataset helpful to your research, please consider citing:

@article{hu2023gauhuman,
  title={GauHuman: Articulated Gaussian Splatting from Monocular Human Videos},
  author={Hu, Shoukang and Liu, Ziwei},
  journal={arXiv preprint arXiv:},
  year={2023}
}

🗞️ License

Distributed under the S-Lab License. See LICENSE for more information.

🙌 Acknowledgements

This project is built on source codes shared by Gaussian-Splatting, HumanNeRF and Animatable NeRF.