Skip to content
Face swap and 3D alignment from a single image based on PRNet
Branch: master
Clone or download
Latest commit 66831c1 Feb 23, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
FaceConverter.xcodeproj update repos Feb 17, 2019
FaceConverter fix bug, when face fade out from left. Feb 21, 2019
opencv2.framework update repos Feb 17, 2019
README.md Update README.md Feb 23, 2019
libncnn.a Face 3D alignment and Face Swap Jan 17, 2019
prnet.mlmodel "estimateRigidTransform, warpAffine" replaced by "roi and scale" Jan 17, 2019

README.md

C++ implementation of PRNet on iOS

This project contains:

  1. Face reconstruct(Face swap) from only a single image.
  2. Face pose estimate and keypoints(68) in 3D model.

The origin python code with training is in PRNet.

Platfrom and requirements

1. iPhone SE, arm64
2. OpenCV 3.4.0
3. NCNN

Outline

  1. Mtcnn --> resize to 256x256 --> PRNet --> UV,Z --> Render 86906 triangles in 3D --> SeamlessClone
  2. Mtcnn --> resize to 256x256 --> PRNet --> 43867 vertices --> SVD to estimate pose
    Mtcnn --> resize to 256x256 --> PRNet --> 43867 vertices --> pick 68 points for sparse alignment

Application

  1. Face Swap, change the target face image(@"ref.jpg") to your own.
  2. Face 3D pose estimate and face keypoints in 3D.

About the configuration

There's 4 config files in this project.

1. face_ind.txt

This file is the index of [0,65535], which is the region of the WHITE area of the image[256x256] below.

2. uv_kpt_ind.txt

This file is the index of coordinate(x,y) for 68 keypoints refer to face_ind.txt

3. triangles.txt

86906 triangles' vertices index refer to face_ind.txt

4. canonical_vertices.txt

canonical model for pose estimate, there's coordinate (uv,z) of 43867 vertices

Other details

  1. About the Euler angles.

http://www.gregslabaugh.net/publications/euler.pdf

  1. 3D points transformation matrix.

http://nghiaho.com/?page_id=671

  1. Render 3D texture.

http://blackpawn.com/texts/pointinpoly/

You can’t perform that action at this time.