Replies: 6 comments 9 replies
|
What do you have your concurrency set to on that job? Setting that value too high can actually be counterproductive. Also what CPU do you have in the mini pc? It's possible it's just not a very fast cpu. Also there are other things that increase the number of thumbnails to be generated as the job continues, like facial recognition. |
|
An older i5, concurrency set to 2-4 as I don't have many cores :) I think it's related to the thumbnail structure that was chamged a few months ago and I think my migration didn't complete properly as it all works ok up to a point and then it starts lagging like hell. i think that if the migration job is not run properly, or if there are inconsistencies in the db with the thumbnail hashes it enters a loooooong loop where it takes all items one by one and tries to identify existing thumbs, else generates them, but if the file already exists it skips or waits a while... causing such a low speed of job processing. I'm trying now to re-run the migration job and check for any issues in the logs, then i'll re-run the library job and get back with updates. It may be isolated to an improper existing thumbnail structure that's broken... |
|
Update: ran the Migration job successfully then ran the Library job - again finished really fast - but I keep seeing in the log some errors about duplicates (exactly this scenario: #5067 ) |
|
So far it seems that re-running the Migration job fixed the problem with the neverending thumbnails job... Guess there was an inconsistency in the thumbnail folder structure... If all goes well I'll write the conclusion and the discussion can be safely closed |
|
Back to square 1... Logs of microservices node keep on saying it's generating thumbnails (though the filesystem has the same number of files, so no new thumbs generated... or maybe they're overwritten ...) and the constraint violation error in postgres: If you suggest anything, I'm all ears. PS: why does the Storage Template Migration job keep on starting (every file that's parsed by the metadata extraction) ?? Is there any template migration that's required when simply running the Library job ? |
|
Conclusion after some testing: anyway, now i run library refresh and then missing thumbnails job and that seems to do the trick. |
|
@andyxpert I have a problem with thumbnail generation as well, but I am totally new to immich so I might have something wrong with my setup. My external library of ~120,000 photos is mounted over smb, and thubmnail generation never finishes... I tried using tree to count the images in the thumbs folder, and I have seen it go down, but I do not see anything interesting in the logs. |
|
We had a bug a few releases back where external libraries could import generated images as new images and make an infinite feedback loop. You might be affected if you had an import path that includes the If that doesn't apply, it's probably just that generating so many images takes a good deal of time. Images are also added to the queue as other jobs get processed, so it's normal for the waiting number to increase even as jobs finish because there's a steady stream of new jobs being added. If you have an SSD and a beefy CPU and notice they both aren't getting fully utilized, you can set the Conversely, if you're seeing that everything is fully pinned or you're using a lot of swap memory, you may want to pause the machine learning jobs so the other jobs can get processed more efficiently. |
|
Pretty sure I am experiencing this issue. #11040 Or is there good reason my thumbs folder has 150k images in it for 90k photos in my library? |
|
That's normal. A thumbnail and preview image file is generated for each asset, and person thumbnails also exist in that folder. |
I use Home Assistant to monitor Immich Job status through its API
Try clearing the job and restarting the service then restarting the job (for the missing files) |
|
Update: upgraded my mini-PC, now running a 9th gen i5 6 cores with 32 GB RAM. Allocated 2 cores and 8 GB for Immich Services I think there is still a bug in the thumb job which gets fixed/avoided by clearing the job cache and restarting the entire stack (or just the service) Btw, the log is clean no duplicate files or error generating thumbs and the thumbs seem to be constantly generated, so the error might be only on how the remaining job count is calculated (increasing instead of decreasing)
|
|
FYI I'm having the same problem as @andyxpert just above. The jobs page reports 3XX,XXX thumbnails "waiting" and even though it is generating thumbnails at a decent clip, the cue just gets ever-larger. It grew from about 200,000 to around 350,000 in maybe an hour. I have about 250K images but all the thumbnails, except for a batch of around 1000 photos I just added, had previously been generated & working fine. Looking at the immich logs, it looks like it is generating new thumbnails for all sorts of images and then just deleting the previously generated thumbnails for the image. Also - and what brought this to my attention - the thumbnails for the newly added images were not being generated. They just showed up as and X. docker compose down redis followed by docker compose up redis -d got things back on track. It generated missing thumbnails for the new 1000 or so photos in just a few minutes and seems just fine now. |
|
I'm also having this issue. External library of 32k photos is generating its thumbnails from yesterday at full blast of ryzen 5 5600g + 8gb of RAM. I cannot comprehend why every time i look into it's stats, and stop generating and rerun job for missing thumbs it says different number (I'm not talking about differences like few hundreds, but difference between 10k-90k). Speeds I'm getting is 3/4 photos per minute at best. My concurent jobs were changed 3-6 which did nothing. Everything is hosted locally on Unraid through docker compose |
|
Did you enable full-size thumbnail generation? That can slow down processing quite a bit. Otherwise, the speed depends on a lot of things. It could be falling back to swap if there isn't enough ram, it could be very slow disk speed, or it could be that your processor is just that slow (granted a 5600g should be much faster than 3-4 assets per minute). If you have a ton of videos, generating thumbnails for them is also more intensive than for photos. |
Uh oh!
There was an error while loading. Please reload this page.
Hello, I'll try to be brief...
Been using immich for almost a year already and love it.
Recently I'm having an issue when I run the Library job on the external lib (~65k files, mostly photos), after I add some new files to the folder, it takes around half an hour (slow mini-pc) to parse all the files, then it extracts metadata (mostly in parallel) and then it starts generating thumbnails. Nothing off so far.
The problem is that after a while (1-2 hours or so) only the thumbnail job remains active and it seems to be veeeeeeeeery slow.
Slow meaning that it processes around 4-5 items per minute... and having 54k+ files pending it takes close to a week to finish !!
is this normal ?
Why does the thumbnail generation speed drop so much and why does it take that long ?
The microservices node is hogging the RAM, nothing interesting in the logs...
attached a screenshot of the pending taska for the 3 jobs i mentioned - it's over a 12 hour timespan. You can clearly see the spikes when it started the library job (around 10h ago).
What do you guys think ?
All reactions