Accuracy issue

[taewookim]

When I run this model on video, i've noticed the prediction changes pretty wildly from frame to frame, depending on the face dimensions, lighting, position, etc. But as soon as the face becomes closed to the camera, the predictions are more consistent.

Two questions

1. the default 32x32... why was that chosen?
2. could prediction be helped by training / running on larger dimensions? (64x64 or even much larger)

[yu4u]

Hi @taewookim,

1. I simply chose 32x32 for efficiency in development and testing. It is not appropriate for deployment for its poor accuracy as you pointed out.
2. I think the image size should be larger if accuracy matters. The image size 64x64 is used in the sample code in README.md as python3 create_db.py --output data/imdb_db.mat --db imdb --img_size 64. In the original paper [1], the VGGNet is used; the input image size is 224x224.

[1] R. Rothe, R. Timofte, and L. V. Gool, "DEX: Deep EXpectation of apparent age from a single image," ICCV, 2015.

[taewookim]

ah oops.. completely forgot about that. Thank you @yu4u

[taewookim]

By the way, how do you figure out the upper limit on how far this architecture can be taken? I'm sure this is more of a hardware limitation as well as how much time I'm willing to wait on, but are there any guidelines?

[yu4u]

Did you mean how to determine the size of the network?
I like WideResNet with moderate depth (16-22); it provides good trade-offs between accuracy and computational cost.

[taewookim]

@yu4u

I tried changing img_size to 224 and the model is performing really bad. For each of the 3 dense layers (i.e. classifying 3 things), loss was anywhere between 9 and 20 .. and accuracy didnt go anywhere even above 0.25. I didn't even bother finishing 30 epochs (i stopped at 28)

I am re-training with img_size at 64, but do you have any idea where I might have gone wrong and what are some things that I can tweak?

EDIT: Lowering to 32 img_size helped to lower loss ... currently on epoch 7:

Epoch 7/30 - loss: 3.7325
dense_3_loss: 1.4535 - dense_1_loss: 0.2372 - dense_2_loss: 1.5244
dense_3_acc: 0.3509 - dense_1_acc: 0.9 - dense_2_acc: 0.4345

[yu4u]

Currently, my implementation is not suitable for larger sizes of images (e.g. 224) for several reasons: 1) it saves and loads all raw images; it requires large amount of memory, 2) the stride of the network is only 4 (before pool (8, 8)) because small image size is assumed.

If you use the image size 224, it seems good idea to use officially supported pre-trained models:
https://keras.io/ja/applications/

[taewookim]

so what's the largest possible that would you'd recommend w/your implementation? im on Google compute w/12gb VRAM K80 GPU w/ 30gb RAM

[yu4u]

I think 64x64 is good enough because there seem to be many low resolution images in the dataset.

[darshansharma]

Hi, I tried this project with your pretrained model with default depth and width but it is giving the wrong prediction. for example it is predicting age 43 F of a 80-year female lady. It is predicting wrong on each and every image.

[yu4u]

Please confirm that your backend and channel order are consistent with the pretrained model.

[darshansharma]

I am using your pretrained model and my hardware specs are different from your one. Is that an issue?

[yu4u]

What is your backend?

[ghost]

Hi, I'd like to ask you where I can see the calculation of this model? because its accuracy is bad from 0.2 to 0.4

[yu4u]

See:

https://github.com/yu4u/age-gender-estimation/blob/master/demo.py#L101-L105

[ghost]

Thank you for your reply. But I mean train and test(validation) accuracy.
When I set the learning rate = 0.001 by all epochs, It would be trained for train dataset. but val_loss increased so there was an over-fitting(did not decrease val_loss)

[yu4u]

The validation accuracy is automatically calculated by Keras using validation_data=(X_test, [y_test_g, y_test_a]).
Overfitting would be alleviated by data augmentation.