HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models

Yifan Yang · Dong Liu · Shuhai Zhang · Zeshuai Deng . Zixiong Huang . Mingkui Tan

CVPR 2024

Table of Contents

1. Introduction to HiLo
2. Running Demo
3. Training and testing
4. Citation

Introduction-to-CR-NeRF

Pipeline of HiLo

• If you want to Train & Evaluate, please check installation.md to prepare environment, required models and extra data. Please check dataset.md to prepare THuman2.0 and CAPE dataset, see Training and testing to train and benchmark HiLo using the prepared datasets.

• If you want to Running Demo, please see Running Demo.

Giving a RGB image of clothed human, with our HiLo, you will get:

• image:
  • with the normals of smpl and cloth
• mesh:
  • with the 3d objects of smpl, reconstructed and refined cloth
• video:
  • showing the reconstructed human from all angles

Running Demo

#Set $in_dir, $out_dir and cuda devices in command/infer.sh
bash command/infer.sh

The reconstructed results (mesh, image, video) will be in path "{$out_dir}".

Training and testing

If you want to train and test the model

#Set experiment name and cuda devices in train_and_test.sh 
bash command/train_and_test.sh

If you only want to test the model

#Set the experiment name to match the training name, and set cuda devices in test_only.sh  
bash command/test_only.sh

Citation

@misc{yang2024hilo,
      title={HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models}, 
      author={Yifan Yang and Dong Liu and Shuhai Zhang and Zeshuai Deng and Zixiong Huang and Mingkui Tan},
      year={2024},
      eprint={2404.04876},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgments

Here are some great resources we benefit from:

• ICON
• ECON
• THuman2.0