face recognition percent

[lavrenkov-sketch]

Describe the bug
I uploaded about 1000 faces for recognition, after which I get the following recognition results for all of them they are always great, but the percentages are negligible, the same photos are used for testing as in the database

Expected behavior
example recognition
curl -X POST "http://localhost:8000/api/v1/recognize" -H "Content-Type: multipart/form-data" -H "x-api-key: a1ed2873-c353-40ff-8e30-c24b0c499c04" -F file=@22.jpg -F prediction_count=1
example to add face
curl -X POST "http://localhost:8000/api/v1/faces/?subject=lavrenkov&retrain=force" -H "Content-Type: multipart/form-data" -H "x-api-key: a1ed2873-c353-40ff-8e30-c24b0c499c04" -F file=@1.jpg

Log docker
compreface-ui | 172.18.0.1 - - [16/Sep/2020:13:57:15 +0000] "DELETE /api/v1/faces/?subject=lavrenkov&retrain=force HTTP/1.1" 400 70 "-" "curl/7.72.0"
compreface-api | {"application":"FRS-ML","@timestamp":"2020-09-16T13:57:21.687+00:00","@Version":"1","message":"Model a1ed2873-c353-40ff-8e30-c24b0c499c04 unlocked, models locked : 0","logger_name":"com.exadel.frs.core.trainservice.component.FaceClassifierLockManager","thread_name":"http-nio-8080-exec-1","level":"DEBUG","level_value":10000}
compreface-api | {"application":"FRS-ML","@timestamp":"2020-09-16T13:57:21.688+00:00","@Version":"1","message":"Model a1ed2873-c353-40ff-8e30-c24b0c499c04 locked, models locked : 1","logger_name":"com.exadel.frs.core.trainservice.component.FaceClassifierLockManager","thread_name":"http-nio-8080-exec-1","level":"DEBUG","level_value":10000}
compreface-ui | 172.18.0.1 - - [16/Sep/2020:13:57:22 +0000] "DELETE /api/v1/faces/?subject=lavrenkov&retrain=force HTTP/1.1" 200 99 "-" "curl/7.72.0"
compreface-api | {"application":"FRS-ML","@timestamp":"2020-09-16T13:57:23.031+00:00","@Version":"1","message":"L-BFGS: initial function value: 6458.66950","logger_name":"smile.math.BFGS","thread_name":"a1ed2873-c353-40ff-8e30-c24b0c499c04","level":"INFO","level_value":20000}
compreface-api | {"application":"FRS-ML","@timestamp":"2020-09-16T13:57:23.997+00:00","@Version":"1","message":"L-BFGS converges on gradient after 1 iterations: 5626.84107","logger_name":"smile.math.BFGS","thread_name":"a1ed2873-c353-40ff-8e30-c24b0c499c04","level":"INFO","level_value":20000}
compreface-api | {"application":"FRS-ML","@timestamp":"2020-09-16T13:57:24.267+00:00","@Version":"1","message":"Model a1ed2873-c353-40ff-8e30-c24b0c499c04 unlocked, models locked : 0","logger_name":"com.exadel.frs.core.trainservice.component.FaceClassifierLockManager","thread_name":"a1ed2873-c353-40ff-8e30-c24b0c499c04","level":"DEBUG","level_value":10000}
compreface-api | {"application":"FRS-ML","@timestamp":"2020-09-16T13:57:24.268+00:00","@Version":"1","message":"Model a1ed2873-c353-40ff-8e30-c24b0c499c04 unlocked, models locked : 0","logger_name":"com.exadel.frs.core.trainservice.component.FaceClassifierLockManager","thread_name":"a1ed2873-c353-40ff-8e30-c24b0c499c04","level":"DEBUG","level_value":10000}

compreface-ui | 172.18.0.1 - - [16/Sep/2020:13:58:02 +0000] "POST /api/v1/recognize HTTP/1.1" 200 292 "-" "curl/7.72.0"
compreface-ui | 172.18.0.1 - - [16/Sep/2020:14:00:33 +0000] "POST /api/v1/recognize HTTP/1.1" 200 290 "-" "curl/7.72.0"

Additional context
results recognition:

"result" : [ {
"box" : {
"probability" : 0.99777,
"x_max" : 471,
"y_max" : 462,
"x_min" : 278,
"y_min" : 216
},
"faces" : [ {
"similarity" : 0.00202,
"face_name" : "0d41f5e020f793f7f64392912ec6792819981201"
} ]
} ]

{
"result" : [ {
"box" : {
"probability" : 0.99988,
"x_max" : 223,
"y_max" : 260,
"x_min" : 60,
"y_min" : 59
},
"faces" : [ {
"similarity" : 0.00251,
"face_name" : "0e08144e69c99d86beb78bbdb7a142cf19730925"
} ]
} ]

[pospielov]

Hi @lavrenkov-sketch,
The problem is with logistic regression - the more classes you have, the lower numbers it return.
The possible solution is to normalize them. But there still be problem that depending on how many faces you have you will receive different numbers of similarity.
To fix this globally we decided to use another approach - instead of logistic regression we will calculate similarity using euclidean distance. Then similarity won't change when number of faces change. The accuracy will be the same or better. The time of recognition could decrease, but slightly(this operation took 32ms for 1000 faces on my laptop).
The work is still in progress, but I hope we will be able to release it during next month.

[pospielov]

@lavrenkov-sketch we just released our first version and we would like to receive some feedback.
We are interested in how you found us, what features you are interested in and at least in some information about the problem you are trying to solve
If you would like to help us in any way, please join the chat:
https://gitter.im/CompreFace/community