test vgg-face with 4096 dimensional outputs on LFW dataset

[serengil]

Currently, our vgg-face model has 2622 outputs. But this is the output for classification model of vgg-face model. In original paper, they are using its early layer with 4096 dimensions for descriptor. Test this and compare with exisiting model.

[serengil]

If we change, VGGFace.py as

    # from
    # vgg_face_descriptor = Model(inputs=model.layers[0].input, outputs=model.layers[-2].output)
    # to
    base_model_output = Sequential()
    base_model_output = Convolution2D(4096, (1, 1), name="predictions")(model.layers[-4].output)
    base_model_output = Flatten()(base_model_output)
    vgg_face_descriptor = Model(inputs=model.input, outputs=base_model_output)

and update distance.py as

"VGG-Face": {"cosine": 0.66, "euclidean": 0.61, "euclidean_l2": 1.12}

thresholds are found from this tutorial.

cosine and euclidean_l2 metrics are giving good results but euclidean is giving bad result as shown in attachment.

[serengil]

on the other hand, model with 2622 dimensional vectors has the following distributions. euclidean is more stable, but cosine and euclidean l2 was more stable in 4096d one.

[serengil]

Finally, if we replace the vgg descriptor as

    base_model_output = Sequential()
    base_model_output = Convolution2D(4096, (1, 1), name="predictions")(model.layers[-4].output)
    base_model_output = Flatten()(base_model_output)
    base_model_output = Activation("softmax")(base_model_output)
    vgg_face_descriptor = Model(inputs=model.input, outputs=base_model_output)

(Just added softmax layer in addition to first experiment)

We have these results:

So, dropping softmax layer makes it more stable.

[serengil]

• try 4096 dimensional model without softmax layer on LFW dataset.
• if cosine and euclidean_l2 overperforms with this form, then overwrite euclidean with euclidean_l2 for vgg-face model.

[serengil]

When I do perform a test for VGG-Face and mtcnn detector

Model	Distance Metric	LFW Score
2622d	euclidean_l2	90.3
2622d	cosine	90.3
4096d	euclidean_l2	96.0
4096d	cosine	96.0

So, using its early layer obviously contributed the score

btw, the best thresholds are 0.6975578183956332 for cosine and 1.186702879413616 for euclidean_l2.

[serengil]

To avoid the failure in euclidean distance metric, what if we add normalization layer to the descriptor? I do not want to overwrite euclidean to euclidean_l2 when model is set to VGG.

base_model_output = Sequential()
    base_model_output = Convolution2D(4096, (1, 1), name="predictions")(model.layers[-4].output)
    base_model_output = Flatten()(base_model_output)
    base_model_output = Lambda(lambda x: K.l2_normalize(x, axis=1), name="norm_layer")(
        base_model_output
    )
    vgg_face_descriptor = Model(inputs=model.input, outputs=base_model_output)

distribution graph on deepface unit test items according to this tutorial becomes stable for euclidean (because it would be equivalent to euclidean_l2), and negative impact on cosine and euclidean_l2.

now, testing all metrics on LFW.

[serengil]

To sum up, (model_name ="VGG-Face", detector_backend = "mtcnn", align = True, enforce_detection = False) configuration is tested on LFW dataset's test samples (1000 instances distributed 50% 50% for same person and different persons);

• In the 4096d one, there is NO softmax layer and there is a l2 normalization layer.

distance metric	2622d	4096d
cosine	90.3	96.0
euclidean	82.2	96.0
euclidean_l2	90.3	96.0

and these are the thresholds:

cosine 0.7077406336745886
euclidean 1.1897399682146557
euclidean_l2 1.1897399998945895

So, doing this action increases the model performance 6% o 14%!

[serengil]

now, it is creating different embeddings in every run. we should get rid of the Convolution2D in the descriptor because it generates random weights for that layer!

base_model_output = Sequential()
    base_model_output = Flatten()(model.layers[-5].output)
    base_model_output = Lambda(lambda x: K.l2_normalize(x, axis=1), name="norm_layer")(
        base_model_output
    )
    vgg_face_descriptor = Model(inputs=model.input, outputs=base_model_output)

[serengil]

With the new structure, same accuracy values gotten

distance metric	2622d	4096d
cosine	90.3	96.0
euclidean	82.2	96.0
euclidean_l2	90.3	96.0

for these thresholds

metric	threshold
cosine	0.6848793279466694
euclidean	1.1735069839807788
euclidean_l2	1.1735069853005953

[serengil]

Closed with PR - #948

[serengil]

vgg-face is the default facial recognition model. so, this improvement will affect every user of deepface.