Training params

[joaootavio93]

Can you describe the parameters format? I noticed that the param_all_norm.pkl file contains 62 extracted parameters of each image from you dataset. By the FaceProfiling code (http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/HPEN/main.htm) I generated a dataset containing my own profiles, but I didn't understand how to create the pkl file using FaceProfiling output variables.

[julioalvess]

Up

[cleardusk]

Sorry for delayed reply. 62=40+10+12, which consists of 40-d shape param, 10-d expression param with 12-d pose param (3x4 matrix form) successively.
If the FaceProfiling output variables consist of shape, expression and pose param, you can build the pkl as follows. The origin dimension of shape param is 199, the expression is 29, you can just select the first 40 of shape and 10 of expression. For pose param, re-calculate them in matrix form with one scale factor f.
The related code will be released in the future as posted in #16, thanks for your attention.

[joaootavio93]

Thank you @cleardusk !
Just two more issues:

• I think the expression param is 29, don't you?
• Is the f scale factor the same generated by the FaceProfiling output?

[cleardusk]

Sorry for my mistake, the dimension expression param is exactly 29. The f scale factor is one of the fitting params, depending on the input image. If the reconstructed 3d face is right, the f scale factor should be right too.

[joaootavio93]

Great! Thank you again!

[julioalvess]

@cleardusk for pose param, do I have to convert to rotation matrix and join the result to the translation array in order to obtain a 3x4 final matrix (consequently the 12-d pose param)?

[cleardusk]

I recommend using the matrix form, and normalize them with mean and std.

[darchonyzx]

@cleardusk @julioalvess @joaootavio93 well, I'm still confused, I just reshape the 12-d pose param to 3x4 matrix form, then decomposite it to scale factor, rotation matrix and translation. Use the Rotation matrix to calculate pitch,yaw and roll, did I do something wrong?

[darchonyzx]

May I ask the 12-d pose param is 3x4 camera matrix? The paper is said that 6-d pose parameters [f, pitch, yaw, roll, t2dx, t2dy].

[cnaaq]

same as @darchonyzx
Also, can somebody please give me a reference to understand more about FaceProfiling output variables and what those 199 shape params and 29 expression params are about? Because I don't really understand why 40 first params out of 199 are selected for this project...

[darchonyzx]

@cnaaq You could refer the main.py and the python files it imports, and you will know how to use it. But as the owner @cleardusk puts it the dlib face detecter may not be useful for the large pose face as 90 angle.
I test the pre-train model under the Multiple Face dataset, and there is a deviation for the 90 face. it works fine for the 75 ~ -75

[darchonyzx]

@cnaaq the former 12 param of output is the pose param, the last column is the offset ,the former 3x3 is the rotation.

[cleardusk]

@darchonyzx Dlib has another version of detector trained by CNN, but I had no chance to validate it. face_recognition project has related code and details. While I recommend using published detector framework based on CNN like MTCNN, SSH or your private detector for large pose face.

[cleardusk]

@cnaaq darchonyzx is right, main.py covers all the details.

[cnaaq]

@darchonyzx @cleardusk
Many thanks for your answers.
I still have two questions:
1. I can infer that offset (3 x 1 vector) is the offset of a fixed points (like tip of the nose) according to the center (or edge) of the image. but I don't understand why it should be 3D vector? We are dealing with 2D images.
2. As mentioned in this paper 'A 3D Face Model for Pose and Illumination Invariant Face Recognition', the focus is on neutral faces. According to this, I don't understand where 29 expression params and 10 params selected out of it come from.
Thanks again.

[cleardusk]

1. homogeneous coordinates.
2. 199-d and 29-d are PCA coefficient params, the first 40-d and 10-d params are enough to represent 3D face.

[seefun]

How can I get the 12-d pose param? Pose param is 6-d in the paper? @cleardusk

[ZHJNCUT]

How to convert the pose parameters (seven dimensions) of the 300W-LP data set into the parameters of 12 dimensions in your code? Could you please teach me how to achieve it?

[ZHJNCUT]

何实现它吗？

Have you solved it yet?

[ZHJNCUT]

@cnaaq输出的前12个参数是位姿参数，最后一列是偏移量，前3x3是旋转。

Hi, I was wondering, is there any connection between it and the Pose_Para (7-dimensional parameter) in the 300W-LP dataset?

[gfzwytc]

Hi, I was wondering, is there any connection between it and the Pose_Para (7-dimensional parameter) in the 300W-LP dataset?

Hello, have you found the solution to this problem?