Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
anxiangsir committed Jan 13, 2021
1 parent 757e9ec commit 3c5e7bc
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 13 deletions.
4 changes: 2 additions & 2 deletions recognition/partial_fc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Partial FC is a distributed deep learning training framework for face recognitio
[Partial FC](https://arxiv.org/abs/2010.05222)
- [Largest Face Recognition Dataset: **Glint360k**](#Glint360K)
- [Docker](#Docker)
- [Performance On Million Identities](#Benchmark)
- [Performance On Million Identities](#Benchmark)
- [FAQ](#FAQ)
- [Citation](#Citation)

Expand Down Expand Up @@ -67,7 +67,7 @@ Use [unpack_glint360k.py](./unpack_glint360k.py) to unpack.
- [x] [**Baidu Drive**](https://pan.baidu.com/s/1sd9ZRsV2c_dWHW84kz1P1Q) (code:befi)
- [x] [**Google Drive**](https://drive.google.com/drive/folders/1WLjDzEs1wC1K1jxDHNJ7dhEmQ3rOOILl?usp=sharing)

| Framework | backbone | sample_rate | IJBC@e4 | IFRT@e6 |
| Framework | backbone | negative class centers sample_rate | IJBC@e4 | IFRT@e6 |
| :--- | :--- | :--- | :--- | :--- |
| mxnet | [R100](https://drive.google.com/drive/folders/1YPqIkOZWrmbli4GWfMJO2b0yiiZ7UCsP?usp=sharing) |1.0|97.3|-|
| mxnet | [R100](https://drive.google.com/drive/folders/1-gF5sDwNoRcjwmpPSTNLpaZJi5N91BvL?usp=sharing) |0.1|97.3|-|
Expand Down
28 changes: 27 additions & 1 deletion recognition/partial_fc/mxnet/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,30 @@
## [中文版本请点击这里](./README_CN.md)
## Speed Up Training
![Image text](https://github.com/nttstar/insightface-resources/blob/master/images/partial_fc.png)

### 1. The classification layer model is parallel
Class centers are evenly distributed across different GPUs. It only takes three communications to complete
loss-free Softmax calculations.

#### 1. Synchronization of features
Make sure each GPU has all the GPU features on it, as is shown in `AllGather(x_i)`.

#### 2. Synchronization of denominator of the softmax function
We can first calculate the local sum of each GPU, and then compute the global sum through communication, as is shown
in `Allreduce(sum(exp(logits_i)))`

#### 3. Synchronization the gradients of feature
The gradient of logits can be calculated independently, so is the gradient of the feature. finally, we collect all the
gradients on GPU and send them back to backbone, as is shown in `Allreduce(deta(X))`

### 2. Softmax approximate

Just a subset of class centers can approximate softmax's computation(positive class centers must in these class centers),
this can be done with the following code:
```python
centers_p = func_positive(label) # select the positive class centers by the label of the sample
centers_n = func_negative(centers_p) # negative class centers are randomly sampled after excluding positive classes
centers_final = concat(centers_n, centers_p) # class centers that participate in softmax calculations
```

## Train
### 1.Requirements
Expand Down
10 changes: 0 additions & 10 deletions recognition/partial_fc/pytorch/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,5 @@
# Parital FC

## TODO

- [x] **No BUG** Sampling
- [ ] Pytorch Experiments (Glint360k, 1.0/0.1)
- [ ] Mixed precision training
- [ ] Pipeline Parallel
- [ ] Checkpoint
- [ ] Docker
- [ ] A Wonderful Documents

## Results
We employ ResNet100 as the backbone.

Expand Down

0 comments on commit 3c5e7bc

Please sign in to comment.