New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How was trained the model? #5

Open
affromero opened this Issue Feb 22, 2016 · 25 comments

Comments

Projects
None yet
5 participants
@affromero
Copy link

affromero commented Feb 22, 2016

Nice model you have got! Thanks for sharing it.

I wonder what kind of data you fed into the network in order to train, I mean, What database do you use to get so many keypoints and also 3d pose?

As you use a face detector, do you perform face cropping before feeding the data into the network?

Could I get more information about the training stage? Would it work for vgg-network?

Another last thing, why did you transform the prediction into this: predictpoints = predictpoints * vgg_height/2 + vgg_width/2 How the labels were normalized?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Jul 7, 2016

  1. i use a software called intraface, it take 49 points ( those 49 points are from 68 ground truth points) and can generate 3d pose.
  2. the face detector is dlib, put the cropped face ( resize to 224, 224) in to network
  3. the label normalize is to make the label range in ( - 1, 1 )。 by ( x - 224/2) /( 224 / 2) (as 224 is the width)
@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Jul 16, 2016

I use the software called intraface. but is only take 12 facial landmarks.The GUI version of intraface is not available for taking 49 points. Which version of intraface used in your training stage?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Jul 17, 2016

This is the intraface I used, you can download from here: https://pan.baidu.com/s/1jIHFSQa.

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Oct 16, 2016

The intraface can only take 49 points.How do you process your training data for detecting 68 points?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Oct 17, 2016

select 49 points from 68 points, you can see the intraface code.

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Oct 17, 2016

I see the intraface code. In the intraface code, it define an object called fa by 'INTRAFACE::FaceAlignment fa(detectionModel, detectionModel, &xxd);'. I see this code 'fa.Detect(frame,*max_element(faces.begin(),faces.end(),compareRect),X0,score) != INTRAFACE::IF_OK' and i print the size of X0. The result shows figure 49. How do i change this code to generate 68 points?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Oct 18, 2016

@Magicyu-1990 intraface predicts 49 facial points and uses this 49 points to genenrate 3d head pose.
I use intraface only to generate 3d head pose ground truth labels.
I use 68points labels from ibug: http://ibug.doc.ic.ac.uk/resources/facial-point-annotations/

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Nov 27, 2016

Hi!I used intraface to generate 3d head pose ground truth labels for 300w face dataset. But i got some errors. In the 300W face dataset, the intraface can not to generate the 3d head pose for some pictures. How do you deal with this problems?
issues2
issues3

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Nov 28, 2016

@Magicyu-1990 intraface could not generate the 3D head pose because of the opencv face detection failure. I ignore these failure images.

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Nov 28, 2016

@qiexing I got confused. If i ignore these failure images, i worry about the number of pictures in the training set. The 300W face dataset has only 300 indoor pictures and 300 outdoor pictures. Training this network will be over-fitting in 300W dataset. How do you tackle these problems?

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Nov 28, 2016

@qiexing I use data augmentation to increase the 300W face dataset with rotation,slide and blur.How do you handle with keypoints labels with the image rotation?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Nov 29, 2016

@Magicyu-1990 The 300 W data is 3000+。 If you do rotation, slide on the images, the position of the landmarks should be transformed respectively. I use data augmentation to enlarge the dataset to 30000+ training images.

@XiaXuehai

This comment has been minimized.

Copy link

XiaXuehai commented Dec 1, 2016

@qiexing The 300 W data is 3000+.Does it include the dataset like Helen and IFPW etc.
Unzip the 300-w.I just got only 300 indoor pictures and 300 outdoor pictures.
11

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Dec 1, 2016

@qiexing I used face detector to detect face and crop the face area to 224*224. But how can i deal with the facial landmarks labels from the original image? The image was cropped to the small size and the landmarks labels are not corresponding to the cropped image. Can you share your valuable experiences for this?
github

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Dec 1, 2016

@XiaXuehai Yes, I used LFPW, Helen, Afw, Ibug. All annotated with 68 points.

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Dec 1, 2016

@Magicyu-1990 . For example. if face bbox is (20, 30, 100, 110) . (20,30) is the left top point, (100, 110) is the right bottom point. The original landmark is (45, 55) , then in the cropped image, it should be (45-20, 55 - 30).
I think the the best way to check if the transformed landmarks are correct is to : display the cropped image and draw transformed landmarks.

@XiaXuehai

This comment has been minimized.

Copy link

XiaXuehai commented Dec 2, 2016

@qiexing I have a question. Using intraface‘s face landmark and pose as ground truth is correct?And if we can get the pose by the intraface correctly, why we still use cnn?

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Dec 12, 2016

@qiexing Why do you normalize the ground truth of facial landmarks to the range of( - 1, 1 )? I see you training code.Your networks has two Loss function. Did you use the multi-task leaning in your training?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Dec 13, 2016

@Magicyu-1990 it's a normalization which will help to learn the landmark positions. Yes, it's multi-task learning.

@sunmiaobo

This comment has been minimized.

Copy link

sunmiaobo commented Jan 13, 2017

How do I use interface, ubuntu can yao, a corresponding document?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Jan 14, 2017

@beita-lab , you mean the intraface document or something else ?

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Mar 21, 2017

@qiexing I see your code to transform the pose prediction into this: predictpose[i] = pose_prediction * 50. When you prepared your training data, How did you normalize the head pose label get from the interface?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Mar 22, 2017

@Magicyu-1990 Most of the head pose of images detected by intraface are ranging in (-50, 50). I do the normalization, so they can ranging in (-1, 1).

@Magicyu-1990

This comment has been minimized.

Copy link

Magicyu-1990 commented Mar 24, 2017

@qiexing Thank you for your kindest help! If i do random rotation on images with angles in (-5,5),(-10,10),how do i calculate the corresponding landmarks with the rotation image?

@qiexing

This comment has been minimized.

Copy link
Owner

qiexing commented Mar 24, 2017

@Magicyu-1990 do the corresponding rotation on the landmarks. Actually, the image rotation is the point rotation + interpolation.

@kartk kartk referenced this issue Jun 29, 2018

Open

poselayer * 50 #10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment