Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Face training is ineffective #879

Open
2 of 3 tasks
docwisdom opened this issue Jun 8, 2023 · 6 comments
Open
2 of 3 tasks

Face training is ineffective #879

docwisdom opened this issue Jun 8, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@docwisdom
Copy link

docwisdom commented Jun 8, 2023

πŸ› Bug Report

  • πŸ“ I've Included a ZIP file containing my librephotos log files
  • ❌ I have looked for similar issues (including closed ones)
  • 🎬 (If applicable) I've provided pictures or links to videos that clearly demonstrate the issue

πŸ“ Description of issue:

I have done considerable work "training" the face recognition model by associating hundreds of photos with dozens of people. When I run the train faces function, it does not infer any new photos for my people even though I have thousands more photos of those people in my library.

πŸ” How can we reproduce it:

Associate a set of photos to a person, then click Train in the library drop down menu

Please provide additional information:

  • πŸ’» Operating system: Unraid
  • βš™ Architecture (x86 or ARM): x86
  • πŸ”’ Librephotos version: I dont know where to find this
  • πŸ“Έ Librephotos installation method (Docker, Kubernetes, .deb, etc.): Docker/Unraid https://hub.docker.com/r/reallibrephotos/singleton
    • πŸ‹ If Docker or Kubernets, provide docker-compose image tag:
  • πŸ“ How is you picture library mounted (Local file system (Type), NFS, SMB, etc.): local
  • ☁ If you are virtualizing librephotos, Virtualization platform (Proxmox, Xen, HyperV, etc.):
    logs.zip
@docwisdom docwisdom added the bug Something isn't working label Jun 8, 2023
@scepterus
Copy link

Can confirm, after the first big batch, it rarely finds new matches for faces. It could be a size of the pool it's trained on for a specific face that exceeds the limits, or it could be the general limit to the size of the training data it can go through to match.

Haven't looked at the logic in the code, but since I also run docker, that's what I think the cause is. Also, if it is, maybe instead of having one major pool of face data, we can specify a pool for each face and have it iterate those pools when training.

@Rihcus
Copy link

Rihcus commented Sep 7, 2023

Can confirm, after the first big batch, it rarely finds new matches for faces. It could be a size of the pool it's trained on for a specific face that exceeds the limits, or it could be the general limit to the size of the training data it can go through to match.

Haven't looked at the logic in the code, but since I also run docker, that's what I think the cause is. Also, if it is, maybe instead of having one major pool of face data, we can specify a pool for each face and have it iterate those pools when training.

By any chance did you run into a bug where new faces are tagged as unkown? I tried training/tagging 200 faces but now new similar faces are getting tagged as unkown (faces are in clear lighting conditions, looking at camera).

Also not sure if this is usefull but:

On my first scan I used hog
Then second/3rd scan used cnn

  • midway 3rd scan I changed it from cnn back to hog (taking to long)

I also ended up with a rouge worker problem (ended up fixing by docker compose down setting the volume folder to a blank folder and letting the stuck process quite, then again docker compose down change env volume back to origional).

After this quick fix I ended up with the face scan getting stuck (library scan worked)

image

image

image

Doing a full rescan (via web ui) seemed to fix the face scan stuck issue but I noticed the inference system (using hog) isn't working.

Not sure if this overall is similar to your issue just curious if any similarities are present.

@scepterus
Copy link

I recently found out about cnn, and that has done a better job (with some fine-tuning) at finding faces, but it still hits a wall. No matter how many photos a person has, at some point it stops training on those photos and finding new ones that match.

@Rihcus
Copy link

Rihcus commented Sep 7, 2023

I recently found out about cnn, and that has done a better job (with some fine-tuning) at finding faces, but it still hits a wall. No matter how many photos a person has, at some point it stops training on those photos and finding new ones that match.

I wonder if this convergence warning (1000) limit is related.
Screen Shot 2023-09-07 at 11 48 29 AM

I switched back to hog but I'm not sure if the act of switching between hog and cnn corrupted the training data.

@scepterus
Copy link

As I learned in another issue, that might give you duplicate faces if you switch between them. as for that limit, looks like it, we need @derneuere to tell us for sure though.

@derneuere
Copy link
Member

Face Detection, Face Recognition, Face Classification and Face Clustering are all different things.

  1. Face Detection finds faces in pictures. Here you can choose between HOG and CNN. It does not change any following steps. It looks like @Rihcus changed the file permission of his thumbnails, which is why the system can't create preview pictures for the faces and fails. Please open a separate issue for that with steps to reproduce.

  2. Face Recognition creates a 512 dimension embedding for a given face. It makes them comparable with each other. Right now we only have the one method and should be good enough.

  3. Face Clustering does not take into account any previously attained knowledge, like already labelled persons. We switched to this system because we do not need any supervision from the user, leading to a better out-of-the-box experience. Clustering only changes, when you either delete / add faces or if you change the parameters in the setting. This is the original issue, he expects that it takes labelled persons into account, which it does not.

  4. Face Classification is how we get the confidence values. This system learns from labelled persons, but currently only does do the confidence values. In the future we will add a mode, where it will match to only persons and not clusters, which is what you guys seem to expect. There seems to be in @Rihcus system an issue, where he has too many clusters. Without any additional information, I can't reproduce that.

Tldr: Play around with the settings for clustering and delete bad faces when you train.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants