DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras (ICCV 2021)

Yang Zheng*, Ruizhi Shao*, Yuxiang Zhang, Tao Yu, Zerong Zheng, Qionghai Dai, Yebin Liu.

This repository contains the official pytorch implementation of ”DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras“.

News

• [2021/9/26] We added more scans to MultiHuman dataset. You can use MultiHuman to train/fine-tune our model or your own models!
• [2021/9/18] MultiHuman dataset for evaluation purpose is available!

Requirements

• PyTorch
• torchvision
• trimesh
• numpy
• matplotlib
• PIL
• skimage
• tqdm
• cv2
• json
• taichi==0.6.39 or 0.7.15
• taichi_three
• taichi_glsl==0.0.10

Pretrained Model

We have provided pretrained model in One Drive

Training

To train DeepMultiCap model, please run the following code.

sh render_dataset.sh
sh train.sh

Demo

Run the following code to inference on the MultiHuman dataset.

sh render_two.sh
sh demo.sh

Citation

@inproceedings{zheng2021deepmulticap,
title={DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras},
author={Zheng, Yang and Shao, Ruizhi and Zhang, Yuxiang and Yu, Tao and Zheng, Zerong and Dai, Qionghai and Liu, Yebin},
booktitle={IEEE Conference on Computer Vision (ICCV 2021)},
year={2021},
}