Unsatisfied results tested on customized images

[BIGJUN777]

Hello

I got some non-ideal results when testing the model on other images.

1. As shown in the following image, there are some distinct boundaries on the resulting image. Any idea on this?

   

   When diving into the code, I found you applied Face Detection before PRNet (See here). Because the image I input has been cropped, I commented those codes related to Face Detection in process() function (See below). However, the result turned out to be worse. It seems this Face Detection is necessary. Would you provide me with more explanation on this?

   

   

2. When testing on my device (Tesla V100), it took several seconds to process an image, which is really time-consuming. Is this normal? If so, why will it be such time-consuming (it seems the model is not really complex)?

Looking forward to your reply.
Thanks.

[thaoshibe]

Thanks for your interest in our work. I'm happy to help!

1. About the boundaries
   Currently, we employed the pre-trained model of PRNet. As PRNet is not a 3D full-face model, it is expected that there will be artifacts in boundaries (Fig. 1). For better results, I suggest:
   1.1. Using image blending technique
   1.2. Employ other 3D full-face models. You can easily use any other 3D full-face models to replace PRNet in our framework.

Fig. 1. Reconstructed face from PRNet

2. About face detection
   Using face detection and cropped face ensure the quality of the PRNet (See more Issues about that here). So yes, you can comment out those codes, but it will affect the performance of the 3D face model, subsequently cause unpleasant results (as you experienced).

3. Testing speed
   I tested on a single RTX 1080Ti, running time ~ 1-2 seconds per image. The slowest part of the code is the 3D parts L34-36. I'd recommend you to check whether the code is currently run on GPU or not (As maybe tensorflow-gpu is not working).
   After that, further details to speed up the 3D model parts can be found on PRNet's repo.

Feel free to ask if you have other questions ( ❛ᴗ❛ )

[BIGJUN777]

Thank you for your detailed as well as useful reply.

For the testing speed, as you mentioned, there is something wrong with the TensorFlow-GPU in my env. Will fix it later.