Skip to content

Latest commit

 

History

History
592 lines (287 loc) · 19.8 KB

3D Face Papers.md

File metadata and controls

592 lines (287 loc) · 19.8 KB

3D Face

Surveys & Doctoral Thesis

  • Face Image Analysis using a Multiple Features Fitting Strategy(2005, Basel)

  • 3D Face Modelling for 2D+3D Face Recognition(2007, Surrey)

  • Image Based 3D Face Reconstruction: A Survey(IJIG2009, Georgios Stylianou, Andreas Lanitis, EUC, CUT)

    early 3D facial acquisition approaches

  • animation reconstruction of deformable surfaces(2010, Hao Li, ETHz)

  • Inverse Rendering of Faces with a 3D Morphable Model(2012, Oswald Aldrian, York)

  • Digital Geometry Processing Theory and Applications(2012, Kun Zhou, Zhengjiang, 中文)

  • State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications

    State of the Art on 3D Reconstruction with RGB-D Cameras(EG2018, MZ, CT, MPI, Stanford, TUM, Disney, Technicolor, UEN) [talks]

Papers & Codes

Reconstruction&3D Alignment&Correspondences

1998 - 2015

  • A Morphable Model For The Synthesis Of 3D Faces

    (SIGGRAPH1998, V BlanzT Vetter , MPI)

    3dmm,analysis-by-synthesis(cascaded, coarse to fine, using texture information), mid-detail

  • Efficient, Robust and Accurate Fitting of a 3D Morphable Model

    (ICCV2003, S Romdhani, T Vetter , Basel)

    3dmm, fitting Algorithm needs: Efficient, Robust, Accurate, Automatic. Mid-detail

  • Estimating 3D Shape and Texture Using Pixel Intensity, Edges, Specular Highlights, Texture Constraints and a Prior

    (CVPR2005, S Romdhani, T Vetter , Basel)

    3dmm, multiple features

  • A 3D Face Model for Pose and Illumination Invariant Face Recognition

    (AVSS2009, Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. , Basel) [data](https://faces.cs.unibas.ch/bfm/)

  • 3D Face Reconstruction from a Single Image Using a Single Reference Face Shape

    (TPAMI2011, I Kemelmacher-Shlizerman, Basri R, UW)

    template, sfs, texture information, mid-detail

  • Face Reconstruction in the Wild

    (ICCV2011, Kemelmacher-Shlizerman I, Seitz S M , UW)

    collection, sparse correspondence, warp template, low-rank approximation(photometric stereo, for expression normalization), mid-detail

  • A FACS Valid 3D Dynamic Action Unit Database with Applications to 3D Dynamic Morphable Facial Modeling

    (ICCV2011, Cosker D, Krumhuber E, Hilton A. , UofSurrey)

    aam, expression

  • Viewing Real-World Faces in 3D

    (ICCV2013, T Hassner, Open U Israel)

    template, sparse correspondence, pose adjustment, depth optimization(SIFT)

  • Improving 3D Face Details based on Normal Map of Hetero-source Images

    (CVPRW2014, Yang, C., Chen, J., Su, N., & Su, G. , Tsinghua University)

  • Total Moving Face Reconstruction

    (LNCS2014, Suwajanakorn S, Kemelmacher-Shlizerman I, Seitz S M. , Washington)

    video(collections), template, average shape, pose estimation, 3d flow(correspondence), refinement, high-detail

  • FaceWarehouse: a 3D Facial Expression Database for Visual Computing

    (VCG2014, Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K., Zhejiang) [data]

  • Intrinsic Face Image Decomposition with Human Face Priors

    (ECCV2014, Li C, Zhou K, Lin S , Zhejiang)

  • Fitting 3D Morphable Models using Local Features

    (ICIP2015, Huber, P., Feng, Z. H., Christmas, W., Kittler, J., & Ratsch, M, Surrey)

    sparse correspondence, 3dmm, regression

  • What Makes Tom Hanks Look Like Tom Hanks

    (ICCV2015, Suwajanakorn S, Seitz S M, Kemelmacher-Shlizerman I. , Washington)

    collection, template, average model, 3D flow, correspondence, deformation vector, TPS, expression sililarity weighted, high-frequency details, Laplacian pyramid

  • Unconstrained Realtime Facial Performance Capture

    (CVPR2015, Hsieh, P. L., Ma, C., Yu, J., & Li, H. , USC)

    video,image collections, occlusion, segmentation, landmarks

  • Unconstrained 3D Face Reconstruction

    (CVPR2015, Roth, J., Tong, Y., & Liu, X, MSU)

    collection, sparse correspondence(landmarks), template, photometric stereo(SVD), matrix completion,

  • Pose-Invariant 3D Face Alignment

    (ICCV2015, Jourabloo, A., & Liu, X., MSU)

    alignment, dense correspondence, visibility, cascaded regressor, 3DPDM.

  • Discriminative 3D Morphable Model Fitting

    (CVPR2015)

2016

#CVPR

  • Large-pose Face Alignment via CNN-based Dense 3D Model Fitting

    (CVPR2016, Jourabloo, A., & Liu, X., MSU)

    alignment, 3dmm

  • Automated 3D Face Reconstruction from Multiple Images using Quality Measures

    (CVPR2016, Piotraschke, M., & Blanz, V , Siegen)

  • A Robust Multilinear Model Learning Framework for 3D Faces

    (CVPR2016, Bolkart, T., & Wuhrer, S., Saarland)

  • Face Alignment Across Large Poses: A 3D Solution

    (CVPR2016, Zhu, X., Lei, Z., Liu, X., Shi, H., & Li, S. Z. , MSU, CASIA)

  • Adaptive 3D Face Reconstruction from Unconstrained Photo Collections

    (CVPR2016, Roth, J., Tong, Y., & Liu, X, MSU)

    landmarks, 3dmm, coarse-to-fine,photometric stereo, time: 7 minutes

  • Augmented Blendshapes for Real-time Simultaneous 3D Head Modeling and Facial Motion Capture

    (CVPR2016, Thomas, D., & Taniguchi, R. I. , Kyushu University)

  • A 3D Morphable Model learnt from 10,000 faces

    (CVPR2016, Booth, J., Roussos, A., Zafeiriou, S., Ponniah, A., & Dunaway, D., ICL)

#ECCV

  • Joint Face Alignment and 3D Face Reconstruction

    (ECCV2016, Liu, F., Zeng, D., Zhao, Q., & Liu, X, MSU, Sichuan U)

    alignment, landmarks

  • Real-Time Facial Segmentation and Performance Capture from RGB Input

    (ECCV2016, Saito, S., Li, T., & Li, H. , USC)

    occlusions, tracking

#Others

  • 3D Face Reconstruction by Learning from Synthetic Data

    (3DV2016, Richardson, E., Sela, M., & Kimmel, R, IIT)

    3dmm, cnn, regress 3dmm parameters, sfs

  • Face Reconstruction on Mobile Devices Using a Height Map Shape Model and Fast Regularization

    (3DV2016, Maninchedda, F., Häne, C., Oswald, M. R., & Pollefeys, M., ETH)

  • A Multiresolution 3D Morphable Face Model and Fitting Framework [code]

    (IJCV2016, Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, P., Christmas, W. J., ... & Kittler, J. ,Surrey)

  • Rapid Photorealistic Blendshape Modeling from RGB-D Sensors

    (2016, USC)

  • 3D Face Reconstruction with Region Based Best Fit Blending Using Mobile Phone for Virtual Reality Based Social Media

    (2016, Anbarjafari, G., Haamer, R. E., Lusi, I., Tikk, T., & Valgma, L. , Turkey)

    landmarks, uv texture, region

2017

#CVPR

  • 3D Face Morphable Models “In-the-Wild”

    (CVPR2017, Booth, J., Antonakos, E., Ploumpis, S., Trigeorgis, G., Panagakis, Y., & Zafeiriou, S., ICL)

    3dmm, register, UV

  • Face Normals “in-the-wild” using Fully Convolutional Networks

    (CVPR2017, Trigeorgis, G., Snape, P., Kokkinos, I., & Zafeiriou, S. , ICL)

  • Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network

    (CVPR2017, Tran, A. T., Hassner, T., Masi, I., & Medioni, G. , USC)

  • Fast 3D Reconstruction of Faces with Glasses

    (CVPR2017, Maninchedda, F., Oswald, M. R., & Pollefeys, M., ETH)

  • DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

    (CVPR2017, Güler, R. A., Trigeorgis, G., Antonakos, E., Snape, P., Zafeiriou, S., & Kokkinos, I. , ICL)

    dense correspondence, uv

  • Learning Detailed Face Reconstruction from a Single Image

    (CVPR2017, Richardson, E., Sela, M., Or-El, R., & Kimmel, R. , Washington)

  • End-to-end 3D face reconstruction with deep neural networks

    (CVPR2017, Dou, P., Shah, S. K., & Kakadiaris, I. A., UofHouston)

    3dmm, dl, directly learn 3dmm parameters

  • A Generative Model for Depth-based Robust 3D Facial Pose Tracking

    (CVPR2017, Cai, L. S. J., Pavlovic, T. J. C. V., & Ngan, K. N. , CUHK)

    occlusions

#ICCV

  • 3D Morphable Models as Spatial Transformer Networks

    (ICCV2017, Bas, A., Huber, P., Smith, W. A., Awais, M., & Kittler, J. , York, Surrey )

    dl, cnn, uv texture, landmarks, stn, 3dmm

  • Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses

    (ICCV2017, Bhagavatula, C., Zhu, C., Luu, K., & Savvides, M., CMU)

  • Pose-Invariant Face Alignment with a Single CNN

    (ICCV2017, Jourabloo, A., Ye, M., Liu, X., & Ren, L. , MSU)

    alignment, 3dmm, dl, cnn

  • Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression code

    (ICCV2017, Jackson, A. S., Bulat, A., Argyriou, V., & Tzimiropoulos, G. , Nottingham)

    end-to-end, 3dmm, dl, cnn, landmarks, voxel

  • Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

    (ICCV2017, Sela, M., Richardson, E., & Kimmel, R. , IIT)

  • MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

    (ICCV2017, Tewari, A., Zollhöfer, M., Kim, H., Garrido, P., Bernard, F., Pérez, P., & Theobalt, C. , MPI)

  • Dense Face Alignment code

    (ICCVW2017, Liu, Y., Jourabloo, A., Ren, W., & Liu, X. , MSU)

  • Realtime Dynamic 3D Facial Reconstruction for Monocular Video In-the-Wild

    (ICCVW2017)

  • Learning Dense Facial Correspondences in Unconstrained Images

    (ICCV2017, Yu, R., Saito, S., Li, H., Ceylan, D., & Li, H.  , USC)

    dense correspondence

#others

  • Large Scale 3D Morphable Models [data](https://faces.cs.unibas.ch/bfm/)

    (IJCV2017, Booth, J., Roussos, A., Ponniah, A., Dunaway, D., & Zafeiriou, S. , ICL)

    for alignment, template, cnn, dl, sparse correspondence, landmarks, tps warping

  • What does 2D geometric information really tell us about 3D face shape? (2017, Bas, A., & Smith, W. A )

    shape from landmarks, shape from contours

  • Pix2Face: Direct 3D Face Model Estimation

    (2017)

    dense correspondence, 3dmm

  • 3D Face Reconstruction with Geometry Details from a Single Image

    (TIP2017, Jiang, L., Zhang, J., Deng, B., Li, H., & Liu, L. , USC, USTC)

    coarse-to-fine, landmarks, corrective deformatio, sfs

2018

#CVPR

  • Unsupervised Training for 3D Morphable Model Regression code

    (CVPR2018, Genova, K., Cole, F., Maschinot, A., Sarna, A., Vlasic, D., & Freeman, W. T , Google)

  • 4DFAB: A Large Scale 4D Database for Facial Expression Analysis and Biometric Applications data

    (CVPR2018, Cheng, S., Kotsia, I., Pantic, M., & Zafeiriou, S., ICL)

  • Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

    (CVPR2018, Cao, X., Chen, Z., Chen, A., Chen, X., Li, S., & Yu, J.  , shanghaitech)

    5 input images, 3dmm, shadow processing, light calibration, photometric stereo, denoising

  • Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition

    (CVPR2018, Liu, F., Zhu, R., Zeng, D., Zhao, Q., & Liu, X. , MSU, Sichuan)

  • Mesoscopic Facial Geometry Inference Using Deep Neural Networks

    (CVPR2018, Hao Li, USC)

    high-detail, dl, scan, uv texture, displacement

  • Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz

    (CVPR2018, Tewari, A., Zollhöfer, M., Garrido, P., Bernard, F., Kim, H., Pérez, P., & Theobalt, C., MPI)

  • SfSNet : Learning Shape, Reflectance and Illuminance of Faces in the Wild

    (CVPR2018, Sengupta, S., Kanazawa, A., Castillo, C. D., & Jacobs, D.  , Maryland, UCB )

  • Probabilistic Joint Face-Skull Modelling for Facial Reconstruction

    (CVPR2018, Madsen, D., Lüthi, M., Schneider, A., & Vetter, T. , Basel)

  • Alive Caricature from 2D to 3D

    (CVPR2018, Wu, Q., Zhang, J., Lai, Y. K., Zheng, J., & Cai, J, USTC)

  • Nonlinear 3D Face Morphable Model

    (CVPR2018, Tran, L., & Liu, X. , MSU)

  • InverseFaceNet: Deep Monocular Inverse Face Rendering

    (CVPR2018, Kim, H., Zollhöfer, M., Tewari, A., Thies, J., Richardt, C., & Theobalt, C. , MPI)

    Self-Supervised Bootstrapping

  • Extreme 3D Face Reconstruction: Looking Past Occlusions

    (CVPR2018, Tran, A. T., Hassner, T., Masi, I., Paz, E., Nirkin, Y., & Medioni, G. , USC)

  • Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

    (CVPR2018, best student paper, Joo, H., Simon, T., & Sheikh, Y. , CMU)

  • Modeling Facial Geometry using Compositional VAEs

    (CVPR2018, Bagautdinov, T., Wu, C., Saragih, J., Fua, P., & Sheikh, Y., EPEL, FRL )

#ECCV

  • 3D Face Reconstruction from Light Field Images: A Model-free Approach

    (ECCV2018, Feng, M., Gilani, S. Z., Wang, Y., & Mian, A., Western Australia, Hunan)

    epopolar plane images

  • Generating 3D Faces using Convolutional Mesh Autoencoders [code]

    (ECCV2018, Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J., MPI)

  • Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network [code]

    (ECCV2018, Feng, Y., Wu, F., Shao, X., Wang, Y., & Zhou, X., SJTU)

#others

  • Morphable Face Models - An Open Framework

    (FG2018, Gerig, T., Morel-Forster, A., Blumer, C., Egger, B., Luthi, M., Schönborn, S., & Vetter, T. , Basel)

  • CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images [data&code]

    (TPAMI2018, Yudong Guo, Juyong Zhang, Jianfei Cai, Boyi Jiang, Jianmin Zheng, USTC)

  • Multilinear Autoencoder for 3D Face Model Learning(WACV 2018, Universite Grenoble Alpes (LJK), France)

    3d scan to registered mesh. dl. height map

  • On Face Segmentation, Face Swapping, and Face Perception(AFGR,2018, HT)

  • Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild(FG2018) data

#arxiv

  • Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition(2017, Feng Liu, Xiaoming Liu)

  • Convolutional Point-set Representation: A Convolutional Bridge Between a Densely Annotated Image and 3D Face Alignment(20180317)

  • Unsupervised Depth Estimation, 3D Face Rotation and Replacement(20180325)

Production-level Reconstruction

more in computer graphics

  • High-Quality Single-Shot Capture of Facial Geometry(TOG2010, ETHZ, Disney)

    cg, high-detail,stereo system, calibration, surface refinement, normal direction, mesoscopic

  • Multiview Face Capture using Polarized Spherical Gradient Illumination(TOG2011)

    image collecitons

  • High-Quality Passive Facial Performance Capture using Anchor Frames(SIGGRAPH2011, ETHZ, Disney)

    cg, stereo,anchor frame, tracking, mesh progration, physical movement, motion estimation, refinement

  • Lightweight binocular facial perfor- mance capture under uncontrolled lighting(TOG2012, MPI)

    cg, high-detail, stereo, template,flow,data term, geometry term, smoothness term, mesh tracking, motion refinement, shape refinement, sfs

  • Reconstructing Detailed Dynamic Face Geometry from Monocular Video(TOG2013, MPI)

    cg, dynamic, high-detail, blend model, sparse correspondence, dense correspondence(appearance matching, LBP), pose estimation , shape refinement, sfs

  • 3D Shape Regression for Real-time Facial Animation(TOG2013, ZJU)

  • Real-Time High-Fidelity Facial Performance Capture (TOG2015, ZJU)

    cg, landmarks, optical flow, train a regressor to learn detail

  • Dynamic 3D Avatar Creation from Hand-held Video Input(TOG2015, EPEL)

    cg, dynamic, mobile, high-detail, avatar, 3dmm,sparse correspondence, eye mesh, tracking, refinement, sfs, detail map

  • Reconstruction of Personalized 3D Face Rigs from Monocular Video(TOG2016, MPI)

    parametric shape prior, coarse-scale reconstruction, fine-scale(sfs), coase->medium->fine, 3dmm, corrective

  • Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks(ASCA2017, USC)

  • Multi-View Stereo on Consistent Face Topology(EG2017, USC)

    cg, high-detail, landmarks, template, pose estimation, refinement

  • Avatar Digitization From a Single Image For Real-Time Rendering(SIGGRAPH Asia 2017, USC)

    cg, avatar, segmentation, head, hair, 3DMM, landmarks, texture completion

  • Learning a model of facial shape and expression from 4D scans(TOG2017, USC, MPI)

  • DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling(SIGGRAPH2017)

  • High-Fidelity Facial Reflectance and Geometry Inference From an Unconstrained Image(SIGGRAPH2018, USC)

Texture

3D-aid texture generation/ UV texture completion
Keys: GAN

  • Face Synthesis from Facial Identity Features(CVPR2017, google)

    3dmm, dl, landmarks

  • Photorealistic Facial Texture Inference Using Deep Neural Networks(CVPR2017, Hao Li, USC)

    texture completion

  • UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition(CVPR2018, SZ, ICL)

    gan, 3dmm, uv texture completion

  • Multi-Attribute Robust Component Analysis for Facial UV Maps(2017, SZ, ICL)

  • Realistic Dynamic Facial Textures from a Single Image using GANs(CVPR2017, Hao Li, USC, DeepMind)

  • Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model(2018)

  • Side Information for Face Completion: a Robust PCA Approach(20180120, SZ, ICL)

Transfer&Reenactment(Applications)

  • Face Transfer with Multilinear Models (SIGGRAPH2005)

    Cartesian product(ID x EX x VI)

  • Online Modeling For Realtime Facial Animation(TOG2013)

    rgbd, blendshape, corrective field

  • Displaced Dynamic Expression Regression for Real-time Facial Tracking and Animation(SIGGRAPH2014)

  • Real-time Expression Transfer for Facial Reenactment(SIGGRAPH AISA 2015)

  • Face2Face: Real-time Face Capture and Reenactment of RGB Videos(CVPR2016)

    capture, transfer, 3dmm, landmarks, texture, expression, mouth retrieval

  • Synthesizing Obama: Learning Lip Sync from Audio(SIGGRAPH2017)

  • Deep Video Portrait(SIGGRAPH2018)

  • HeadOn: Real-time Reenactment of Human Portrait Videos(SIGGRAPH2018)

3D-aid 2D face recognition

  • Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification(ECCV2012, Columbia University)

  • Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics(PAMI2006)

  • 3D-aided face recognition robust to expression and pose variations (CVPR2014)

  • Effective 3D based Frontalization for Unconstrained Face Recognition(ICPR2016, MICC, Florence)

  • Effective Face Frontalization in Unconstrained Images(CVPR2015, TH, Israel)

  • Do We Really Need to Collect Millions of Faces for Effective Face Recognition(ECCV2016, TH, USC, Israel)

  • High-Fidelity Pose and Expression Normalization for Face Recognition in the Wild(CVPR2015)

  • When 3D-Aided 2D Face Recognition Meets Deep Learning: An extended UR2D for Pose-Invariant Face Recognition(2017)

  • Towards Large-Pose Face Frontalization in the Wild

  • Fully Automatic Pose-Invariant Face Recognition via 3D Pose Normalization (ICCV2011, Cambridge, MA, USA)

3D face recognition

  • Face Identification across Different Poses and Illuminations with a 3D Morphable Model(Automatic Face and Gesture Recognition2002, VB&TV)

  • Preliminary Face Recognition Grand Challenge Results(2006)

  • expression Invariant 3D Face Recognition with a Morphable Model(FG2008, TV, Basel)

  • Bosphorus Database for 3D Face Analysis(2008)data

  • Robust Learning from Normals for 3D face recognition(ECCV2012, SZ, ICL)

  • Static and dynamic 3D facial expression recognition: A comprehensive survey(IVC2012, SZ, LijunYin)

  • Deep 3D Face Identification(2017, USC)

  • Robust Face Recognition with Deeply Normalized Depth Images (2018)

    depth image(front&neural)

  • Learning from Millions of 3D Scans for Large-scale 3D Face Recognition(CVPR2018, Western Australia)