Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

People: Matching unrecognized faces has become very slow #2182

Closed
lastzero opened this issue Mar 24, 2022 Discussed in #2022 · 43 comments
Closed

People: Matching unrecognized faces has become very slow #2182

lastzero opened this issue Mar 24, 2022 Discussed in #2022 · 43 comments
Assignees
Labels
bug Something isn't working released Available in the stable release

Comments

@lastzero
Copy link
Member

Discussed in #2022

Originally posted by hulmgulm February 8, 2022
Hi,

assigning unrecognized faces got awfully slow since the last release (or the second last, I'm not sure about that). It takes now more than two minutes until I can assign the next face. From looking at the logs, it looks like there are some issues with the data:

2022-02-08 20:16:40 INFO änderungen erfolgreich gespeichert
2022-02-08 20:16:40 INFO faces: merged 77 clusters
2022-02-08 20:16:39 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:16:31 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:16:27 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:16:19 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:16:15 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:16:07 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:16:03 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:15:55 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:15:51 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:15:43 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:15:39 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:15:31 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:15:28 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:15:19 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:15:16 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:15:07 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:15:04 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:14:55 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:14:52 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:14:43 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq
2022-02-08 20:14:40 WARN faces: failed removing merged clusters for subject jr1e4i93ltym0svq
2022-02-08 20:14:31 WARN faces: SAF7A54HA4RCJ2UYGKND7SFCJCJEG62B has ambiguous subject jr1e4i93ltym0svq

It's always the same warning. How do I resolve this issue? Can I do something in the UI or is there something else like an SQL query I can execute (I'm familiar with that)?
I don't want to drop my data, because I spent weeks assigning all faces in my >100k photos.

Thank you!
Andreas

@lastzero lastzero added help wanted Well suited for external contributors! needs-analysis Requires further investigation labels Mar 24, 2022
@lastzero lastzero self-assigned this Mar 24, 2022
@cassava
Copy link

cassava commented Mar 27, 2022

I have the same issue. Just imported 30'000 photos with ~450 new faces, and it's a real crawl to do anything. The whole page just freezes up.

Haven't check the logs yet though.

@lastzero lastzero added the in-progress Somebody is working on this label Mar 28, 2022
@lastzero
Copy link
Member Author

Hello @cassava @hulmgulm @l8gravely :)

I am currently looking into this issue... more information would be helpful to provide you with a solution:

  • Has anyone experiencing this changed config defaults, especially for the index worker interval, facial recognition parameters, or TensorFlow?
  • Which PhotoPrism version(s) are affected? Did you upgrade or is this a new installation? Does it make a difference?
  • Are you using the default MariaDB database version from our official docker-compose.yml config example or are you using a custom configuration / external database?
  • Is it possible that you have (accidentally) assigned the same face to different people (or very similar faces that may not even belong to the same person, e.g. twins)? A sponsor recently had such a case that was difficult to resolve. Once it was resolved, the problems disappeared.

Thank you for taking the time to answer these questions, I really appreciate it. Sorry if you have already posted it somewhere!

@hulmgulm
Copy link

Hi @lastzero,

I've set the following config option
WakeupInterval: 30000
because otherwise PhotoPrism was using 100% CPU 24/7.

I started using PhotoPrism Oct 22nd and updated instantly, each time a new image was available. I think, that this issue started around beginning of February but I'm not sure about that.

At the beginning, I was using an existing MySQL instance for PhotoPrism, until I ran into upgrade issues and you dropped MySQL support. Then I migrated to MariaDB (database dump and import) and am using version 10.7.3 now. I'm using "--innodb-buffer-pool-size=1G" there.

Two weeks after I started this thread, I decided to start nearly from scratch and truncated subject, markers and faces tables and let PhotoPrism do a rescan of the images. At the beginning, it worked like a charm, but soon the issues started to come up again. It does not happen on all face assignments, but at maybe 75%. Currently, it takes about 3 minutes to get these faces saved.

I wrote a tool, which displays me all faces of a given subj_uid, face_id or subj_name. While I noticed one or two wrong assignments (which I corrected in the UI), the other faces are correct. This is a part of the most recent log:

time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with 5XY4C5KHJVHP2XACPRPXND57H5CCKRA5, subject jr7kdtd3u8khnpo5, dist 0.023031"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with Y55MIHW6HG4NLXPHWQHRC6VH4XA2NN4G, subject jr7kdtd3u8khnpo5, dist 0.049753"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with J4V3BXRK4QTOIEMO45N5LT7AQ6XGIQJV, subject jr7kdtd3u8khnpo5, dist 0.043881"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with JSP44VOXH6ERLYVNSXPSJRT3TM2QEJGX, subject jr7kdtd3u8khnpo5, dist 0.034311"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with QMZZVSHVZ6A3N6O4463OKO64SFOZUFG3, subject jr7kdtd3u8khnpo5, dist 0.222249"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with 26ODXCPQ44IEBSWXW5SCJSJ6QX3RDJXR, subject jr7kdtd3u8khnpo5, dist 0.119968"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with 2DLGKEZCMG7AKUZZSSP54OLZOJPYMJBB, subject jr7kdtd3u8khnpo5, dist 0.381513"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with 3SCFX57HZEWZL2GIWC7NNQC4YFV2DQVJ, subject jr7kdtd3u8khnpo5, dist 0.415974"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with 4KAYLP5QYLK3VUGTQQEEZPJAOSYVOETL, subject jr7kdtd3u8khnpo5, dist 0.423640"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with 4WG22ONA2F6T5JLM3M5IYURYFN5OU7V4, subject jr7kdtd3u8khnpo5, dist 0.405420"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with 7BITBQO4HAAKVCQR7MMI5AFN55EOYGZF, subject jr7kdtd3u8khnpo5, dist 0.371248"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with AIN476M4JWKVCBT5ZFYT3T5BGWPVNFJC, subject jr7kdtd3u8khnpo5, dist 0.397838"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with B2C3AJZM4VBS3QKIC7MJY753XFVTUXEX, subject jr7kdtd3u8khnpo5, dist 0.413219"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with BMNM6PK4TEQDPG2ZIQD4DFV26HJEF3UN, subject jr7kdtd3u8khnpo5, dist 0.374699"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with FUXZIOLD6D5D5GTTPDTQMEJUNPHJQPCL, subject jr7kdtd3u8khnpo5, dist 0.396865"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with GW76JBED4JP6B3WN2DTEBE6JER7GFJDV, subject jr7kdtd3u8khnpo5, dist 0.417725"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with H2UOEKZS6XMVEONCZYNYACOTML5W2ACI, subject jr7kdtd3u8khnpo5, dist 0.413470"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with HWELOHQ7G6HAZH6L2PURXABZT4LVEMKE, subject jr7kdtd3u8khnpo5, dist 0.398260"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with JCCDC6FBMWG2CCNXMUFOMVTBEUE4CTXZ, subject jr7kdtd3u8khnpo5, dist 0.365825"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with KZT5RICDFCJZ5GRSEGNLNELWEVXVJOHN, subject jr7kdtd3u8khnpo5, dist 0.394442"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with N77KBX7IDA26FTITNDETWFV24OL7UOTJ, subject jr7kdtd3u8khnpo5, dist 0.337689"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with OYN2NON6DHTIBULJQBASJA5XMWJSICWF, subject jr7kdtd3u8khnpo5, dist 0.399960"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with QWPJ4MA27P4DSORP4GDXTBRORGU6OCGJ, subject jr7kdtd3u8khnpo5, dist 0.373058"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with SI27JEYL6P5TA7EBWAGWALQICKKZQQ4Z, subject jr7kdtd3u8khnpo5, dist 0.381921"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with TPUAFR4BH5BNVKH4BIAWYO3SJYCK45N6, subject jr7kdtd3u8khnpo5, dist 0.417240"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with TQQV7UYABKRR6EUQ4ETRJY7YLDYLT2Y3, subject jr7kdtd3u8khnpo5, dist 0.365432"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with TXQHPIRP5TGKT3BL6UN45G7X25NOP24C, subject jr7kdtd3u8khnpo5, dist 0.336326"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with TZM66ZWBOUGUEXKYVDTH6QTIKNXBSLW3, subject jr7kdtd3u8khnpo5, dist 0.357090"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with US6DUXFHCF666YFGLT6NMFTJLDYEY7ZI, subject jr7kdtd3u8khnpo5, dist 0.356153"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with VVPTFYIBMMMS4FAJL6KB2O7TZSV524NJ, subject jr7kdtd3u8khnpo5, dist 0.414445"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with XEF52KCD4KCP5AA2ACTKX734SIFAJQD5, subject jr7kdtd3u8khnpo5, dist 0.422834"
time="2022-03-28T18:10:43Z" level=debug msg="faces: can merge MDCNKKHLDGZKYRCJZ7EDZYBBZTUXSTF4 with YTQ6DMHUWOTPEPJPIMKZIQPW6CKYMIPZ, subject jr7kdtd3u8khnpo5, dist 0.387638"
time="2022-03-28T18:10:43Z" level=warning msg="faces: JSP44VOXH6ERLYVNSXPSJRT3TM2QEJGX has ambiguous subject jr7kdtd3u8khnpo5"
time="2022-03-28T18:10:47Z" level=warning msg="faces: failed removing merged clusters for subject jr7kdtd3u8khnpo5"

I reviewed all of the mentioned faces as well as the subject and they're all of my oldest daughter.
I would love to find out, which photo is exactly causing these issues.

One thing I have to mention is, that I have many Asian friends and in the beginning it happened a lot that PhotoPrism was assigning the same person to completely different faces (e.g. middle aged male to 10 year old girl). I corrected all of these assignments.

Besides truncating the three tables mentioned above, I did only make changes to the data with PhotoPrism UI.

Maybe this helps, or maybe you can tell me what more I can do, e.g. finding the faces which prevent PhotoPrism from merging them.

Thank you!
Andreas

@l8gravely
Copy link

I wonder if part of my problem is that I'm indexing and searching for photos of my kids, from ages 0 to 19, so they all get assigned to one kid, but the faces are radically different as the kid gets older. Or not radically to my eye, but obviously quite a bit different.

It would be great is there were some better tools to dump what the DB thinks is out there, since with my 70.000+ pictures, it is slow at times, and not really obvious what's going on.

I'm using the default docker-compose.yml file, though I do pull it at points and reload it. So I'm using the default Mysql/mariadb setup (not sure which honestly) and I did try tweaking the memory size of the DB to see if that would improve things. But it's not clear where the slowdown is, if it's in the DB, or in the core code.

Can we turn on slow query logs maybe? Or could we get the logging to start putting some time stamps at the start and end of the update transaction so we can get an idea of what's going on here?

@lastzero
Copy link
Member Author

We developed and tested face recognition assuming the default 15 minute background worker interval, so it is completely untested with very long intervals and not designed for that.

The worker makes sure that the faces get properly clustered/optimized in the background so that the number doesn't get too large to process. The first/next time it runs, it will cause a high server load until all faces, people, and related photos have been updated. The longer you wait, the more CPU is needed and the longer it takes.

30000s is about 8 hours if I calculated it correctly, which generally won't cause any problems (except for less frequently updated photo titles) unless you use face recognition very actively - especially by manually assigning names to many non-clustered faces or correcting wrong matches.

Even our old Core i3 server (running multiple instances and lots of other services simultaneously) rarely has a high CPU load. Maybe on the first day, when lots of faces had to be optimized and clustered, I don't remember any irregularities - but was also too busy to check the system parameters, to be honest.

From our User Manual at https://docs.photoprism.app/user-guide/organize/people/:

  • Recognition does not start until your library has been fully scanned. Searching and updating faces temporarily causes a high CPU load and may take a while, depending on your hardware and the number of images you have.
  • Existing clusters are automatically optimized in the background, for example, when new faces are detected, you have reported a bad match, or new files are added to your library.

Also note that while PhotoPrism can be installed on most consumer NAS devices, such hardware generally does not provide an ideal user experience for compute-intensive applications such as machine learning and face clustering:

https://docs.photoprism.app/getting-started/troubleshooting/performance/#server-cpu

@srett
Copy link
Contributor

srett commented Mar 29, 2022

@l8gravely Did you manually tag a lot of individual faces, i.e. not just the "new faces" clusters you see under "People"? That starts to cause performance issues after a while, as it creates lots of small clusters internally.

Maybe an acceptable mitigation for now regarding this would be to skip the "optimize faces" background job if nothing changed during the last execution of that job, and nothing was edited/added by the user. I'm thinking of a really simple implementation with just an in-memory "changed" flag that isn't even persisted over restarts. It should avoid pointless CPU waste when you don't touch your instance for days.

@l8gravely
Copy link

l8gravely commented Mar 29, 2022 via email

@srett
Copy link
Contributor

srett commented Mar 30, 2022

But the performance slowdown is when you are making changes. I don't mind if there's background optimization, though maybe it should be only during midnight to six am (for example) and only when the load on the system is under some other value.

Did you check if it is only slow while the background job is running? If you set the worker interval to 8 hours and wait until the CPU is idle, is assignment still slow?

Is there any way to get a measurement of the number and size of clusters associated with a name?

To see the number of faces per person:
SELECT subj_uid, subj_name, Count(*) AS facecount FROM subjects LEFT JOIN faces USING (subj_uid) GROUP BY subj_uid ORDER BY facecount DESC;

You can then query the individual faces of a subject from the faces table, which has columns like collision_radius and sample_radius. I thought it would be cool to visualize all the faces as a 2D or 3D point cloud to see if that helps identifying potentially problematic faces, but it seems a face is described through 512 floats, and after reading a paper on how to reduce higher-dimensional data for visualization I decided to give up on this idea. :)

Especially in @hulmgulm 's case, if you want to check whether specific SQL statements are responsible for the hangs you have – 3 minutes sounds excessive – try enabling slow query logging in MariaDB with a reasonable setting for slow query time (3+ seconds).

@cassava
Copy link

cassava commented Mar 30, 2022

Hi @lastzero, thanks for taking the time to look at this!

Here's my setup:

  • Image: TrueCharts.org with default configuration
  • System: TrueNAS Scale / 16 threads / 32GB RAM
  • Install: Fresh
  • Pictures: 30000
  • People: ~450
  • Duplicates: 0 tagged
  • Server CPU usage: ~0%
  • Laptop CPU usage in browser: 100% for 1 core
  • Browser: Firefox, current version

The problem was there already when I was assigning the first faces.

Loading tagged people is fast, as soon as I switch to "new", then it slows down significantly
and takes over a minute to just load the pictures. Takes even longer just to get the text cursor pointer so I can name the tagged person.

@cassava
Copy link

cassava commented Mar 30, 2022

Ok, after playing around a little more in the People/New tab, I found the following:

  • I can't really do anything while it's loading pictures
  • It loads about 2-3 pics / second... much slower than with already named people, which is more or less instantaneous.
  • After naming one person, it takes ~5 seconds to save the change, during which the ui is also unresponsive.

@l8gravely
Copy link

l8gravely commented Mar 30, 2022 via email

@srett
Copy link
Contributor

srett commented Mar 30, 2022

Nope, I haven't. What's the process for making these changes since I don't see anything in the docker-compose.yml file.

It's PHOTOPRISM_WAKEUP_INTERVAL; it's controlling all the background jobs (cover updates, titles, ...) that run periodically.

Ok, so I need to fire up a docker shell to get access into the DBs, etc. Still working to wrap my brain around docker and debugging. New tricks, old dog. :-)

Yeah, not the biggest fan of docker either in that regard, but it's not that much harder once you get used to it. For simple debugging the mysql command line client from the container's shell should be sufficient, but you could set up external access if you really wanted to.

@lastzero
Copy link
Member Author

  • I can't really do anything while it's loading pictures
  • It loads about 2-3 pics / second... much slower than with already named people, which is more or less instantaneous.

Sounds like there are no thumbnails yet (face crops from thumbnails), so they are rendered on demand. That would explain why already named people are much faster. Did you delete any, reorganize your library somehow, or is it possible they got lost somehow?

  • After naming one person, it takes ~5 seconds to save the change, during which the ui is also unresponsive.

Did you mention somewhere what database you are using?

@l8gravely
Copy link

l8gravely commented Apr 1, 2022 via email

lastzero added a commit that referenced this issue Apr 3, 2022
May help improve rendering performance on slow devices.
Feedback welcome!
@lastzero
Copy link
Member Author

lastzero commented Apr 3, 2022

I wonder if part of my problem is that I'm indexing and searching for photos of my kids, from ages 0 to 19, so they all get assigned to one kid, but the faces are radically different as the kid gets older.

That's why faces that our current model often confuses - like those of children - should/can not be automatically matched:

After users repeatedly commented that this was not a problem for them, the protection was later disabled to a large extend

When a person's face changes, it's generally not a problem, since it's possible to have more than one matching face. But if the face can't be distinguished, that's a big problem, since it consumes CPU time without producing useful matches - which we may observe here to some extent. It's probably a mix if different issues.

lastzero added a commit that referenced this issue Apr 3, 2022
Adds two unofficial env variables so advanced users can experiment:

1. PHOTOPRISM_FACE_KIDS_DIST=0.6950 (range: 0.1-1.5, -1 to disable)
2. PHOTOPRISM_FACE_IGNORE_DIST=0.86 (range: 0.1-1.5, -1 to disable)
@l8gravely
Copy link

l8gravely commented Apr 3, 2022 via email

@lastzero
Copy link
Member Author

lastzero commented Apr 3, 2022

@l8gravely The latest commit shows the real name in (most?) subject related logs. Didn't have much time to test manually, but the automated tests are all green. So it SHOULD work. Also added a cache, which should help performance.

@l8gravely
Copy link

l8gravely commented Apr 3, 2022 via email

lastzero added a commit that referenced this issue Apr 4, 2022
lastzero added a commit that referenced this issue Apr 4, 2022
Since caching all subject data proved too complex in the time available,
this implementation uses a simple key/value lookup table to cache
subject names and perform backward searches by uid.
@lastzero
Copy link
Member Author

lastzero commented Apr 4, 2022

Since caching all subject data proved too complex in the time available, the new implementation uses a simple key/value lookup table to cache subject names and perform backward searches by UID. Next, I take another look at the face merge function.

lastzero added a commit that referenced this issue Apr 4, 2022
This commit also fixes of other potential issues and improves logging.
@lastzero lastzero changed the title People: Assigning unrecognized faces got awfully slow since the last release People: Matching unrecognized faces has become very slow Apr 4, 2022
@lastzero lastzero added bug Something isn't working and removed in-progress Somebody is working on this labels Apr 4, 2022
@l8gravely
Copy link

l8gravely commented Apr 4, 2022 via email

@l8gravely
Copy link

l8gravely commented Apr 5, 2022 via email

@hulmgulm
Copy link

hulmgulm commented Apr 5, 2022

A few days back, I reset the automatic scan interval to the default be removing the configuration. Now the nearly 24/7 100% were back.

After a few days, I had the impression, that it does not take 3 minutes anymore to save a face but less. I also noticed, that for low quality or too small faces, the issue did not occur.

Started a new preview build for you! Note that we will be traveling until after Easter, so feedback may be slow. Any help with testing these changes is greatly appreciated.

I just updated to that build (photoprism/photoprism:preview) and I don't see any issues anymore. I tagged now about 50 faces and the updates happend within seconds. No errors/warnings in the logs. Whatever you changed, looks like it fixed the issues I was encountering.

@rickysarraf
Copy link
Contributor

THank you. This is a big improvement and gives a decent hint on where the problems might be. It may be worth to document that the information is only logged with the faces audit command. I couldn't get those messages otherwise.

INFO[2022-04-10T15:41:29Z] successful conflict resolution for 'Anshika Raj Aaa Sarraf', face id GNZRHW46GUV363R7NQPMTZTB7TW4N4LI had collisions with other persons 
INFO[2022-04-10T15:45:36Z] face M4LJEMRF7RS53TSIOKR7BDLW24ABNJB6: ambiguous subject at dist 0.073319, Ø 0.810000 from 7 samples, collision Ø 0.783474 
INFO[2022-04-10T15:45:36Z] face M4LJEMRF7RS53TSIOKR7BDLW24ABNJB6: subject 'Aditi Verma' (jr2xm6pt2sqxahy6 auto) 
INFO[2022-04-10T15:45:36Z] face GNZRHW46GUV363R7NQPMTZTB7TW4N4LI: subject 'Anshika Raj Aaa Sarraf' (jr28la51lq9c8zg9 auto) 
INFO[2022-04-10T15:45:36Z] successful conflict resolution for 'Aditi Verma', face id M4LJEMRF7RS53TSIOKR7BDLW24ABNJB6 had collisions with other persons 
INFO[2022-04-10T15:45:36Z] face M4LJEMRF7RS53TSIOKR7BDLW24ABNJB6: ambiguous subject at dist 0.793474, Ø 0.810000 from 7 samples, collision Ø 0.063319 
INFO[2022-04-10T15:45:36Z] face M4LJEMRF7RS53TSIOKR7BDLW24ABNJB6: subject 'Aditi Verma' (jr2xm6pt2sqxahy6 auto) 
INFO[2022-04-10T15:45:36Z] face R2Q2IPV6QI7W37RXVM7NNAJBG5XDUQBC: subject 'Anshika Raj Aaa Sarraf' (jr28la51lq9c8zg9 auto) 
INFO[2022-04-10T15:45:36Z] successful conflict resolution for 'Aditi Verma', face id M4LJEMRF7RS53TSIOKR7BDLW24ABNJB6 had collisions with other persons 
INFO[2022-04-10T15:45:52Z] face ET6RDQS2XCU246LGFTFD6ELO2Q36DR6F: ambiguous subject at dist 0.049647, Ø 0.810000 from 49 samples, collision Ø 0.275798 
INFO[2022-04-10T15:45:52Z] face ET6RDQS2XCU246LGFTFD6ELO2Q36DR6F: subject 'Anuja Soni' (jr2xm9l59657rumn auto) 
INFO[2022-04-10T15:45:52Z] face 6GWIPLND442EPVJL5OUR2ATMFWXZQ42H: subject 'Priyanka Prasad' (jr2xmec27npuk1ge auto) 

@rickysarraf
Copy link
Contributor

One more question please.

How should the below be read/understood ? It says person name conflicts with face XXX subject ' ', both in auto and manual mode.

INFO[2022-04-11T19:20:46Z] marker mr0q0d73f6cavvgq: manual subject 'Anshika Raj Aaa Sarraf' conflicts with face ZSLHBBD2C7SMZFA4OW265OB2HBWCEWZS subject '' 
INFO[2022-04-11T19:20:46Z] marker mr1s5ht2s0znwgdv: auto subject 'Anshika Raj Aaa Sarraf' conflicts with face ZSLHBBD2C7SMZFA4OW265OB2HBWCEWZS subject '' 
INFO[2022-04-11T19:20:46Z] found no orphan face clusters                
INFO[2022-04-11T19:20:46Z] found no orphan people                       
INFO[2022-04-11T19:20:46Z] completed in 3h14m16.330249375s              
INFO[2022-04-11T19:20:46Z] closed database connection                   

@lastzero
Copy link
Member Author

@rickysarraf Could you try again with these changes? I've started a new preview build for testing, available soon.

@rickysarraf
Copy link
Contributor

@rickysarraf Could you try again with these changes? I've started a new preview build for testing, available soon.

Hello @lastzero

With the latest changes, I now get:

INFO[2022-04-14T20:45:12Z] faces: found no ambiguous subjects                                                                                       
WARN[2022-04-14T20:45:12Z] faces: marker mr0prv62ro0ezs7v has invalid face 3BKRFAQ7QHIFALST6G52JUHDR7C252P5 of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0psd4yzp9s5syj has invalid face 3BKRFAQ7QHIFALST6G52JUHDR7C252P5 of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0q4jz1cxwymi1v has invalid face 3BKRFAQ7QHIFALST6G52JUHDR7C252P5 of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr1c7xe1qcdjreq3 has invalid face 3BKRFAQ7QHIFALST6G52JUHDR7C252P5 of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0prlb37913axt5 has invalid face 565FP7NVWZ4NWG2U23Q2QDG2UG6FAS3X of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0pscy2uf72jxso has invalid face 565FP7NVWZ4NWG2U23Q2QDG2UG6FAS3X of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0q4jz2tlyfv4y5 has invalid face 565FP7NVWZ4NWG2U23Q2QDG2UG6FAS3X of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0pr5f354mk4zsz has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0pr952hbq1xmyy has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0prdw1wj0bz54f has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0prgq3fhcedk2z has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0prh4ru3mtrl04 has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0prhv3ldolr05i has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0prkw1elidgaqa has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> () 
WARN[2022-04-14T20:45:12Z] faces: marker mr0ps2yc4yu8a71j has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> ()                 
WARN[2022-04-14T20:45:12Z] faces: marker mr0ps433unftyd62 has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> ()                 
WARN[2022-04-14T20:45:12Z] faces: marker mr0ps8bxq4sq7xa0 has invalid face 5T4BAFUHLK6AF5DMUAQUFNLLMXKJ2S7V of subject <unknown> ()                 

@lastzero
Copy link
Member Author

lastzero commented Apr 15, 2022

This is because a face ID is referenced that apparently no longer exists. Have you tried the photoprism faces audit --fix command (designed to remove invalid data)? You could also try to manually set the affected database fields to an empty value and see if that fixes your issue...

@graciousgrey graciousgrey added released Available in the stable release and removed help wanted Well suited for external contributors! please-test Ready for acceptance test labels May 17, 2022
@l8gravely
Copy link

l8gravely commented Oct 11, 2022 via email

@lastzero
Copy link
Member Author

Well, right now we're working on multi-user and security, so don't expect any improvements in facial recognition. I'm not an octopus developing with 8 keyboards and computers at the same time, and I don't know who else would work on it!? 🤔

@MartinVerges
Copy link

@lastzero facial recognition is one of the most important features in my opinion. I for example paid to support you just because of this single individual feature. Reading this here after searching for a way to speed up the very annoying slowdown makes me sad.

@lastzero
Copy link
Member Author

lastzero commented Mar 5, 2023

Would you please be so kind to check the follow up issue before publicly coming to conclusions? Do you have any idea how much we work and how large our team is? May I ask what you do for a living and how much you earn? Possibly, that makes us sad too.

@lastzero
Copy link
Member Author

lastzero commented Mar 5, 2023

@lastzero
Copy link
Member Author

lastzero commented Mar 5, 2023

$25 one time. How many developers are we supposed to hire from this, Martin?

@MartinVerges
Copy link

What has my company to do with my private pictures of my family?

@lastzero
Copy link
Member Author

lastzero commented Mar 5, 2023

That's all you have to say after publicly posting an unfounded rant? Did you even understand what I wrote above?

@MartinVerges
Copy link

The only person behaving improperly is you. I have only communicated that in my personal opinion, this is an important feature. I've been using this software for only a few days, made an initial payment to you to support the project, and I'm being personally attacked here. Please explain to me why someone who just tried your software and even paid immediately should pay more just because he owns a company?

I am happy to pay for software, and my behavior has clearly shown this. Sorry I gave the honest feedback that this feature is the only reason for me personally to support it. Without the fsce detection, I can use any other software.

@lastzero
Copy link
Member Author

lastzero commented Mar 5, 2023

Let me help you: if you are as rich as it seems, and this is as important as you say, surely you could support us more than with the minimum amount so that we can grow our team and do all the things you think we should do. It's not like we don't want to do that. Also, it might help to communicate a little more politely in public. PhotoPrism is the result of hard work and is anything but sad.

@MartinVerges
Copy link

If you just read above again, you'll find that I'm just unhappy that you guys aren't making any improvements to facial recognition at this time according to your own writing. Here is what you wrote: "don't expect any improvements in facial recognition" and this is sad. Expressing my sadness over this statement is certainly not a rant.

What is the point of the statement "if you are as rich as it seems"? What does that have to do with anything? Do you ask your other sponsors how much money they have? Do sponsors who support you have to sponsor you depending on their wealth or income? I myself write and release open source hardware and software that I need in my private time for my personal hobbies. Never in my life would I think of snorting at someone who supports one of my projects in such a misguided way or even think of using his financial background as a justification.

@lastzero
Copy link
Member Author

lastzero commented Mar 5, 2023

Please check the issue I have linked above. We are working on it.

Edit: We just did not have any resources at that time, which was last year. Read more carefully before coming to conclusions.

@lastzero
Copy link
Member Author

lastzero commented Mar 5, 2023

We kindly ask you to use GitHub Discussions or any of our other community forums next time instead of commenting your personal opinion in GitHub Issues. Thank you very much! ❤️

@ALEEF02
Copy link

ALEEF02 commented Mar 18, 2024

I wrote a tool, which displays me all faces of a given subj_uid, face_id or subj_name. While I noticed one or two wrong assignments (which I corrected in the UI), the other faces are correct. This is a part of the most recent log:

@hulmgulm Can you share with me the source for that tool? I'm trying to improve the clustering on my own set of photos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working released Available in the stable release
Projects
Status: Release 🌈
Development

No branches or pull requests

9 participants