Kinship Representation Learning with Face Componential Relation

[vitalwarley]

• Código: https://github.com/wtnthu/FaCoR
• Paper: https://arxiv.org/pdf/2304.04546.pdf

Chamou-me a atenção por estar entre os melhores resultados na comparação feita na tabela 6 de #50.

[vitalwarley]

1. Pre-Reading

• Abstract Key Points:

  • Problem

    • Research on Kinship Recognition mostly focuses on heuristic designs without considering the spatial correlation between face images. #KinshipVerification #KinshipRecognition#Research

      • What are "heuristic designs" as mentioned in the abstract?

  • Method

    • Face Componential Relation Network (FaCoRNet).

      • What does it do? They aim to learn discriminative kinship representations embedded with the relation information between face components.

      • How does it do?

        • Cross-attention mechanism: learn important facial regions from face components among images.

        • Relation-Guided Contrastive Learning: adapt the loss function by the guidance from cross-attention to learn more discriminative feature representations.

  • Results

    • Outperform previous SOTA by largin margins.

• Figures/Tables Noted:

  • Face Componential Relation Guidance

    

  • Face Componential Relation Network

    

  • SOTA on FIW

    

    • What "Contrastive (naive)" means?

    Qual a diferença entre os backbones ArcFace e AdaFace?

  • Performance by quality-filtered protocol

    

  • Component Analysis

    

    • How can we interpret these improvements? Rel-Guide shows only 0.003, 0.003, and 0.039 improvement.

  • Visual analyis with t-SNE

    

    Insight: see the baby from family 22. What if we manage to cluster individuals by age, then by family? The baby would be proximal to other babies, not adults.

    I am thinking about the notion of Recognizability Index. In kinship we wouldn't have Unrecognizable Identities, but Unverifiable Individuals. A baby is somewhat unverifiable.

    But if we cluster by age, we could complicate the kinship verification, as some kinship types rely on it. It would be useful to know how many errors on present SOTA are due to age. Gender, also, sure.

    What if I pre-train the model from Recognizability Embedding Enhancement for Very Low-Resolution Face Recognition and Quality Estimation, which proposed the Recognizability Index, on the family classification task à lá Achieving Better Kinship Recognition Through Better Baseline? Unrecognizable individuals would cluster? A baby, for example, is a good example of potential unrecognizable individual in the family context.

    The RI aims to improve recognition of low resolution images. This paper here aims, as future work, to incorporate face quality scores into the training process. There is something to do here.

  • Cross-Attention map results

    

• Conclusion Key Points

  • Proposal

    • FaCoRNet is an attention-based model designed for learning correlation between image pairs in terms of face components. Its FaCoR module achieves adaptive learning correlation between image pairs and learns important face components for kinship recognition. It helps against large variations in facial appearance. Cross-attention estimation as a relation indicator guides the regular contrastive loss without the need for heuristic tuning.

      • What heuristic tuning is?

  • Future work

    • Incorporate face quality scores into the training process, aiming to mitigate the issues from low-quality face images.

    • Multi-modal information (e.g., text, metadata) to compensate for the vision-based methods.