Import: Improve logs when another job is running simultaneously

[spaibs]

What does not work as expected?

After importing > 200 pictures, those errors occur:

photoprism-photoprism-1 | time="2021-12-13T20:09:36Z" level=warning msg="metadata: open /photoprism/import/DSC_0820.JPG: no such file or directory in DSC_0820.JPG (parse jpeg)"

photoprism-photoprism-1 | time="2021-12-13T20:09:36Z" level=info msg="media: DSC_0820.JPG was taken at 2016-11-27 13:30:24 +0000 UTC (meta)"

photoprism-photoprism-1 | time="2021-12-13T20:09:36Z" level=info msg="import: moving main jpg file DSC_0820.JPG to 2016/11/20161127_133024_EEEEEEEE.jpg"

photoprism-photoprism-1 | time="2021-12-13T20:09:36Z" level=error msg="open /photoprism/import/DSC_0820.JPG: no such file or directory"

photoprism-photoprism-1 | time="2021-12-13T20:09:36Z" level=error msg="open /photoprism/import/DSC_0820.JPG: no such file or directory"

photoprism-photoprism-1 | time="2021-12-13T20:09:36Z" level=error msg="import: failed copying file to 2016/11/20161127_133024_EEEEEEEE.jpg (open /photoprism/import/DSC_0820.JPG: no such file or directory)"

photoprism-photoprism-1 | time="2021-12-13T20:09:36Z" level=error msg="import: media: 20161127_133024_EEEEEEEE.jpg not found in 2016/11/20161127_133024_EEEEEEEE.jpg"
`

The images will be imported somehow, but errors shouldn't be shown

How can we reproduce it?

Steps to reproduce the behavior:

1. Add 200 images into import-folder
2. docker-compose exec photoprism photoprism import
3. Wait for > 200 images
4. See error

What behavior do you expect?

No error messages should be shown.

What version you are using?

211210-2cb90e7e-Linux-x86_64

[lastzero]

Could be multiple import jobs running at the same time?

[spaibs]

I don't think so. I've tested it on two different systems - same behavior.

[lastzero]

Would be surprised if importing > 200 files suddenly causes problems. This is pretty well tested. Could be the auto indexer or specific filename patterns.

[spaibs]

I've got 400 demo pictures: When I split them into batches of 100 images and import them 4 times, everything works fine. This means the images are not causing the problem.

[spaibs]

I've tried again and the error started at image number 150. It is pretty easy to reproduce.

[spaibs]

Are you able to reproduce the issue?

[lastzero]

Not tonight as it's already 11pm. If you are sure it's a bug, we can add a bug label and try to reproduce it as soon as possible. How many cores do you have? What filesystem? How do filenames look like? Every detail can be helpful.

[spaibs]

I think I found the "issue": After adding the photos via WebDav I had started the importing manually. After a specific time (in my case 150-200 photos) the auto indexer started automatically, so in fact two indexers were running simultaneously.

Improvement: Auto-indexer should check if another indexer is running and should only start if that's not the case.
What do you think?

[lastzero]

That's what I meant. Would need to use lock files for this which then may not get removed when the import crashes, for example because no swap is configured.
Did I understand correctly that the only thing that's bothering you is the log message, which in fact was helpful as you now know about the issue?

[spaibs]

From the user's point of view, it is very confusing when the log is full of errors when indexing. Without your help, I would not have immediately realized that an auto-indexer is also running in the background. The user only sees a lot of errors - how is he supposed to know for sure that all his photos have been indexed without errors even though errors have been displayed?

Maybe you can just give a hint to the running auto-indexer in the case of the error "No such file or directory"?