Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 

Learning-Perspective-Undistortion-of-Portraits

Introduction

Near-range portrait photographs often contain perspective distortion artifacts that bias human perception and challenge both facial recognition and reconstruction techniques. In this project we predict a distortion correction flow map that encodes a per-pixel displacement that removes distortion artifacts when applied to the input face image. Our method also automatically infers missing facial features, i.e. occluded ears caused by strong perspective distortion, with coherent details. Our technique benefits a number of fundamental tasks, significantly improving the accuracy of both face recognition and 3D reconstruction and enables a novel camera calibration technique from a single portrait. Moreover, we also build the first perspective portrait database with a large diversity in identities, expression and poses.

Training Dataset

Our training dataset contains 278 different individuals rendered with randomly sampled focal length, camera views and lighting environment. More details about the training data can be found in our paper.

Testing with real-world portraits

To demonstrate that our system scales well to real-world portraits, we also devised a twocamera beam splitter capture system that would enable simultaneous photography of a subject at two different distances. With carefully geometry and color calibration, the two hardware-synced cameras were able to capture nearly ground truth portraits pairs of real subjects both with and without perspective distortions.

Applications

1. Portrait Undistortion (Demos are coming soon...)

*Fried, Ohad, et al. "Perspective-aware manipulation of portrait photos." ACM Transactions on Graphics (TOG) 35.4 (2016): 128.

2. Run-time Head mounted camera undistortion

Our technique can be applied to HMC facial motion capture.

3. Camera Parameters Estimation (Demos are coming soon...)

Our approach provides the power to accurately estimate camera parameters from a single unconstrained face image (focal length and camera-to-subject distance).

4. Robust Image-based 3D Head Reconstruction

The quality of image-based 3D head reconstruction are heavily relying on the input image. Our approach will remove perspective distortion of the input image, so that the distortion and inaccuracy in the corresponding 3D head will be eliminated.

5. Landmark Detection and Face Verification Enhancement

Our approach greatly enhance the performance of face verification and landmark detection of the near-range portraits. More details can be found in the paper.

Paper [Data] [Code](coming soon)

Citation

If you find our project useful in your research, please consider citing:

@article{Zhao2019learning,
  title={Learning Perspective Undistortion of Portraits},
  author={Zhao, Yajie and Huang, Zeng and Li, Tianye and Chen, Weikai and LeGendre, Chloe and Ren, Xinglei and Xing, Jun and
  Shapiro, Ari and Li, Hao},
  journal={arXiv preprint arXiv:1905.07515},
  year={2019}
}

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published