Facial Recognition re-processes lots of assets

[tbleiker]

The bug

When I click "Missing" on the "Facial Recognition" job, thousands of assets are re-processed. When the job finishes and I click "Missing" again, the same amount of assets are processed again.

The OS that Immich Server is running on

Docker on Debian Bookworm

Version of Immich Server

v1.93.3

Version of Immich Mobile App

v1.93.2

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

-

Your .env content

-

Reproduction steps

1. Open "Administration" and then "Jobs" 
2. Click "Missing" under Facial Recognition

Additional information

No response

[mertalev]

This isn't a bug. "Missing" queues faces with no associated person. It needs to process all of these to see if any are eligible to be grouped into people.

[tbleiker]

I see. This is due to "min recognized faces" set to >1?

This is a little unsatisfactory, because every time I take a picture, the server now processes thousands of images (which takes roughly 15min now, avg. 50% cpu for all my 4 cores). And the number of assets to process potentially grows with every image I take...

[mertalev]

It's possible to reduce the number of faces processed by only checking the faces of new assets, but this would lower the quality of the clustering algorithm.

If you set min faces to 1, the behavior will resemble the old algorithm: all faces will be associated with a person and the clustering for new assets will be quick. The other side of that is that the quality will also be closer to the old algorithm (but still somewhat better).

[tbleiker]

Thx for the fast response.

Maybe, "only checking the faces of new assets" can be provided as option and the user can decide.

For my assets, I cannot see an overall improvement of the new clustering algorithm. Therefore, I would prefer speed (and less ressources) over quality 😄

Edit: Nevertheless, I would like to still use "min faces >1"...

[tbleiker]

Oh, and I realised that the facial recognition job is triggered as well for a new image without detected faces.

Edit: rephrasing...

[mertalev]

That's a good point. We can optimize that to only trigger facial recognition when a face is detected.

[tbleiker]

What do you think, are there any other optimization options? Should we open a discussion topic for this? Or discuss it in Discord?

E.g.: I would expect, that the facial recognition for a new picture with e.g. two faces (which are already clusterd) is fast - given that their points are right in the middle of their already known cluster.

[tbleiker]

I haven't yet thought that completly through (with my limited knowledge... 😄) - but that sounds promising...!

[mertalev]

This change will be in the next release through PR #6634

[tbleiker]

Thx a lot! Do you need some additional testing before the next release? Maybe I can find some time in the next few days...

[mertalev]

That'd be nice! You can try it by changing the image tag in immich-microservices to pr-6634 (probably good to set the server to this too, but not required)

[tbleiker]

I wanted to give it a try... but there is no tag pr-6634 for immich-microservices (immich-server), but for immich-machine-learning and only with cuda, openvino or armnn.

I am tempted to use main for all images... but I am afraid to break something 😛