Using embeddings for Face Recognition task

[mrapplexz]

Hello! I'm getting face embeddings with these steps:

from torchvision.transforms import functional as VF

cut = cv2.resize(cut, (112, 112), interpolation=cv2.INTER_AREA) # resize image to 112x112 px
cut = VF.to_tensor(cut).to(self.device).unsqueeze(0) # convert ndarray to torch.Tensor, send it to GPU and add batch dimension
cut = VF.normalize(cut, [0., 0., 0.], [1., 1., 1.]) # normalize image as in inference/gen_feat.py
embed = self.model(cut) # do forward pass
embed = F.normalize(embed, p=2, dim=1) # L2-normalize embedding vector
embed = embed.squeeze(0).cpu().numpy() # remove batch dimension and convert embedding vector back to ndarray

For face recognition I'm using cosine similarity between two images.
It works pretty good if I look straight into camera and I get 0.7-1.0 similarity, but if I rotate my face left or right, I get something between 0.1-0.3.
Weights used: https://drive.google.com/file/d/1Bd87admxOZvbIOAyTkGEntsEz3fyMt7H/view
Am I doing something wrong? Maybe I should use another metric or not normalize embedding vectors?

[IrvingMeng]

Hi. I guess it's because of the incorrect pre-processing steps.
The images should be aligned into 112x112 pixels based on their 5 facial landmarks.

import cv2
import numpy as np
from skimage import transform as trans
tform = trans.SimilarityTransform()  

arcface_src = np.array(
    [[38.2946, 51.6963], [73.5318, 51.5014], [56.0252, 71.7366],
     [41.5493, 92.3655], [70.7299, 92.2041]],
    dtype=np.float32)
img = cv2.imread(imgname)
tform.estimate(ldmks, arcface_src) #ldmks of current image -- 5 points, same format as arcface_src
M = tform.params[0:2,:]
cut = cv2.warpAffine(img, M, (112, 112))

Here is an example from Insightface