LARNet: Lie Algebra ResNet

Code for our paper：LARNet:Lie Algebra Residual Network for Profile Face Recognition (ICML2021 accepted). Here is the arxiv version.

Directory Structure

LARNet
├─ data
│    ├─ end2end                                        // The directory of training datasets for end2end
│    │                                                 //     e.g. MS1MV2
│    └─ plugin                                         // The directory of requirments for plugin
│           ├─ clean_feature.bin                       //     pretrained face model result, e.g. ArcFace,CosFace, and ours. Here we give a sample (ArcFace-MS1MV2)
│           ├─ pose_estimation.txt                     //     pose prior labels
│           └─ residual_sample_list.txt                //     The list of *clean_feature*
├─ src
│    ├─ end2end                                        // The code for end2end                      
│    │    ├─ End2end_train.py                          //     The training code for end2end
│    │    ├─ ResNet.py                                 //     The original architecture with our subnets' design
│    │    └─ otheroperation.py                         //     The code for loading different training datasets and caffe crop strategy
│    │
│    └─ plugin                                         // The code for plugin
│              ├─ Subnet_ablation.py                   //     The lightweight ablation experiments to show subnets' effectiveness        
│              ├─ Subnet_def.py                        //     The design of our subnets
│              └─ Subnet_train.py                      //     he training code for plugin based on *clean_feature*
│
└─test protocol                                        // The testing datasets                     
     ├─ IJBA
     ├─ CFP

Requirments

• Python 3 with opencv
• pytorch
• ...

Preparations

Datasets

Training:

• MS1MV2: Our main training dataset is MS-Celeb-1M .
• CASIA-WebFace: In some comparative experiments, we use CASIA-WebFace. You can downlaod form Google Drive or Baidu Cloud with the key:y3wj.

Testing:

• IJBA: We use the aligned images provided from Google Drive or Baidu Cloud.
• CFP: The original CFP dataset can be download from Google Drive or Baidu Yun. And we offer a test protocol for further processing.
• ...

Models

• Pretrained model for plugin methods. Because our plugin method does not need to change the structure and parameters of any pretrained model, it can directly use clean features of other methods as inputs, e.g. ArcFace, CosFace, and our results. Here we give a small training sample picked from MS1MV2 randomly and its corresponding name list.
• Pose labels for all methods. We need to obtain the priori pose labels as inputs. Here we also provide a corresponding pose label file.

Training

plugin method

Make sure all files in LARNet/data/plugin are prepared well.

• Train the residual subnet:

cd LARNet/src/plugin
python Subnet_train.py

After this, a model will be saved in the ./plugin_subset.pth

• ablation study for subnets: Make sure ./plugin_subset.pth exists.

cd LARNet/src/plugin
python Subnet_ablation.py

If you want to change the gating control function or the architecture of residual subnet, you can change the functions in the file Subnet_def.py:

def Gating_Control

or

class Res_Subnet

end2end method

Make sure training datasets (e.g. MV1MV2) are prepared well.

• Train the entire model (you can choose the type of ResNet, 18, 34, 50 and more):

cd LARNet/src/end2end
python End2end_train.py

After this, a model will be saved in the --model_dir/checkpoint.pth.tar
If you want to change the gating control function or the architecture of residual subnet, you can change the part in the file ResNet.py.

if self.end2end:
    res_feature = self.fc1(m_feature)
    res_feature = self.relu(res_feature)
    res_feature = self.fc2(res_feature)
    res_feature = self.relu(res_feature)

    angle = angle.view(angle.size(0),1)
    angle = angle.expand_as(res_feature)
            
    feature = angle * res_feature + m_feature
else:
    feature = m_feature

Testing

IJBA

CFP