Problem about 'postprocess.p' output

[Fanziapril]

The code could run successfully, but the output is random. The output of the deep network is good but when doing 'Affine Align', the alignment results between the depth and 3D template are random. The results are different when running the code several times. Sometimes only partial face is reconstructed. I think this may be the depth cannot align well with the template. The 'postprocess.p' is protected so I cannot figure out the problem.

[matansel]

OK. Can you please share the image?

[Fanziapril]

When I run for the first time, the output of the deep network is:

This is good. And During the postprocess function:

This is a good alignment.

And the final result is:

When I run for the second time, the output of the deep network is:


This is slightly different from the first time.

During the postprocess function:

The liagnment between the depth and template is bad, then the final result is partial:

I don't know why this happened... I am running the code on the Ubuntu 14.04 LTS.

[matansel]

So obviously the result of the network is different than what we've got. Since concolutional layers are not scale invariant, it is important to use the same scale of the face in the input image as in the training images. Did you use the same face cropping we used? Maybe the face detector provides different bounding box than ours. Try to continuously increase the size of your bounding box around the face such that the face will take a smaller portion of the image.

[Fanziapril]

Thanks for your quick reply. I am using the exact same face detector(vision.CascadeObjectDetector and all the property is default) as you are. But I am using MATLAB 2017a, maybe the face detection algorithms are different between different MATLAB versions.

I manually cropped the bounding box and try to make it looks like the image showed in your paper(Fig 10).
FaceDetect = vision.CascadeObjectDetector;
BB = step(FaceDetect,img);
scale = 1.1;
BB(1,1) = BB(1,1) - 0.5*(scale-1)*BB(1,3);
%BB(1,2) = BB(1,2) - 0.5*(scale-1)*BB(1,4);
BB(1,3) = scale*BB(1,3);
BB(1,4) = scale*BB(1,4);
imsz = size(img);
This solved the problem.
However, there is another problem: I directly used the cropped image, but still get different final results.
The cropped image is:

first running:

second running:

I think the bounding box should not be the reason. Is there any possible that the output of deep network or postprocess are different when running the code for several times?

[matansel]

Could you please attach the result of the network of the right scale?
The postprocessing step is highly dependent upon the result of the network and usually doesn't fail.

[Fanziapril]

Yes. I listed teo different results when I running the code using the same cropped image:
First time:

Second time:

(From the output of network, there are small differences in the left cheek)
First time:

Second time:

First time:

Second time:

[matansel]

The result of the network is still poor because the scale of the face in the image is different than the paper. Perhaps also the relative position of the face in the image is also set inaccurately. Try to run the face detector of Matlab 2015a if you can.

[Fanziapril]

I've tried Matlab2013a, 2014a, 2016a(I didn't find a 2015a in my lab), could you please send me the cropped image so I can find where my problem is?

[matansel]

The input image to the networks is the following:

And this is these are the outputs you should get:
Correspondence map:

Depth map:

The networks are deterministic, so you should get the exact same outputs.