How can I fit a 3D mesh that was not generated with FLAME topology?

[zfountas]

I have seen in multiple videos of your team using FLAME with existing 3D models, for example Churchill's head in VOCA. I have downloaded such models but the script fit_3D_mesh.py cannot use them as input as they have a different number of vertices and, I suspect, not exactly the same mapping between vertices and parts of the head. Thanks for the help!

[TimoBolkart]

Yes you are right, fitting FLAME to a scan of arbitrary mesh topology requires a differentiable distance function that quantifies the difference between the scan and the FLAME surface (e.g. some scan-to-mesh or mesh-to-scan distance). You would need to replace the used distance function (i.e. the distance to the 2D landmarks, 3D landmarks, or the vertex-to-vertex distance in the provided experiments) with this new distance. Please see the FLAME paper for a more extensive and more formal explanation. Having this said, we sadly don't have such a distance publicly available due to licensing issues.

[TimoBolkart]

We just added some demo code to fit FLAME to a scan of arbitrary topology in the flame-fitting repo.