Local Connectivity-Based Density Estimation for Face Clustering

This repo contains an official implementation for CVPR'23 paper "Local Connectivity-Based Density Estimation for Face Clustering".

Introduction

The proposed clustering method employs density-based clustering, which maintains edges that have higher density. For this purpose, we propose a reliable density estimation algorithm based on local connectivity between K nearest neighbors (KNN). We effectively exclude negative pairs from the KNN graph based on the reliable density while maintaining sufficient positive pairs. Furthermore, we develop a pairwise connectivity estimation network to predict the connectivity of the selected edges.

Requirements

• Python = 3.8.5
• Pytorch = 1.10.2

conda install pytorch==1.10.2 cudatoolkit=11.3 -c pytorch
pip install -r requirements.txt

Dataset

The data directory is constructed as follows:

data
  ├── ms1m
  |    ├── features
  |    |    ├── part1_test.bin
  |    |    ├── ...
  |    |    └── part9.test.bin
  |    ├── labels
  |    |    ├── part1_test.meta
  |    |    ├── ...
  |    |    └── part9.test.meta
  |    └── knns
  |         ├── part1_test
  |         |    └── faiss_k_80.npz
  |         ├── ...
  |         └── part9_test
  |              └── faiss_k_80.npz
  ├── deepfashion
  |    ├── features
  |    |    └── deepfashion_test.bin
  |    ├── labels
  |    |    └── deepfashion_test.meta
  |    └── knns
  |         └── deepfashion_test
  |              └── deepfashion_k40.npz
  └── ijb-b
       ├── 512.fea.npy
       ├── 512.labels.npy
       ├── knn.graph.512.bf.npy
       └── ...

We have used the data from following repositories.

• [MS-Celeb-1M, DeepFashion] https://github.com/yl-1993/learn-to-cluster
• [CASIA/IJB-B] https://github.com/Zhongdao/gcn_clustering

For DeepFashion dataset, we construct kNN graph using faiss.

• Google Drive

Run

To test for each dataset, simply run shell scripts.

sh inference_{dataset_name}.sh

Results on MS-Celeb-1M

1, 3, 5, 7, 9 mean different subset of the clustering benchmark. Detailed settings are on our paper.

Pairwise F-Score

Methods	1	3	5	7	9
CDP	75.02	70.75	69.51	68.62	68.06
L-GCN	78.68	75.83	74.29	73.71	72.99
LTC	85.66	82.41	80.32	78.98	77.87
GCN(V+E)	87.93	84.04	82.10	80.45	79.30
Clusformer	88.20	84.60	82.79	81.03	79.91
STAR-FC	91.97	88.28	86.17	84.70	83.46
Pair-Cls	90.67	86.91	85.06	83.51	82.41
Ada-NETS	92.79	89.33	87.50	85.40	83.99
Chen et al.	93.22	90.51	89.09	87.93	86.94
Ours	94.64	91.90	90.27	88.69	87.35

BCubed F-Score

Methods	1	3	5	7	9
CDP	78.70	75.82	74.58	73.62	72.92
L-GCN	84.37	81.61	80.11	79.33	78.60
LTC	85.52	83.01	81.10	79.84	78.86
GCN(V+E)	86.09	82.84	81.24	80.09	79.25
Clusformer	87.17	84.05	82.30	80.51	79.95
STAR-FC	-	86.26	84.13	82.63	81.47
Pair-Cls	89.54	86.25	84.55	83.49	82.40
Ada-NETS	91.40	87.98	86.03	84.48	83.28
Chen et al.	92.18	89.43	88.00	86.92	86.06
Ours	93.36	90.78	89.28	88.15	87.28

Results on IJB-B

$F_{512}, F_{1024}, F_{1845}$ mean different subset of the clustering benchmark. Detailed settings are on our paper.

Pairwise F-Score

Methods	$F_{512}$	$F_{1024}$	$F_{1845}$
Pair-Cls	84.4	83.3	82.7
Chen et al.	80.8	73.2	59.1
Ours	93.0	92.7	90.8

BCubed F-Score

Methods	$F_{512}$	$F_{1024}$	$F_{1845}$
L-GCN	83.3	83.3	81.4
DANet	83.4	83.3	82.8
Chen et al.	79.6	78.1	76.7
Ours	85.4	85.2	84.8

Results on DeepFashion

Methods	Pairwise F-Score	BCubed F-Score
CDP	28.28	57.83
L-GCN	28.85	58.91
LTC	29.14	59.11
GCN(V+E)	38.47	60.06
Pair-Cls	37.67	62.17
Ada-NETS	39.30	61.05
Chen et al.	40.91	63.61
Ours	41.76	64.56

Acknowledgement

Some codes are based on the publicly available codebase https://github.com/yl-1993/learn-to-cluster.

Citation

@inproceedings{shin2023local,
  title={Local Connectivity-Based Density Estimation for Face Clustering},
  author={Shin, Junho and Lee, Hyo-Jun and Kim, Hyunseop and Baek, Jong-Hyeon and Kim, Daehyun and Koh, Yeong Jun},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2023}
}