Index: Check availability of originals folder before indexing

[graciousgrey]

Some users have problems configuring their originals folder in a way that it is always accessible.
If PhotoPrism indexes when it has no access to the originals folder, all files are flagged as deleted and are not visible anymore in the Web UI.

Acceptance Criteria:

• Before indexing check if originals directory is available, if not throw an error and do not index

--> #2298
--> https://www.reddit.com/r/selfhosted/comments/qafzgz/photoprism_is_deleting_thousands_of_images_as_we/

[gk1089]

Update:
The issue resolved after system restart beyond changing file permissions and docker service restart. Checking of file permissions will indeed be useful for folks like me. I am leaving the old comment intact below though it is now irrelevant.

--
Thank you for putting it in your roadmap. Meanwhile, my issue #2298 remains unresolved. I have uploaded some files in the thread that might be useful. I tried giving 777 permission to everything related to photoprism on my server but that does not seem to be helping.

What else can I do to get it working?
Thanks!

[jackbrown1993]

I faced a similar issue to this when my originals folder (mounted from NAS via NFS) became unreachable and my library went from 140k down to 80k.

Jack.

[lastzero]

Indexing is now skipped and you get a helpful info message saying that your originals folder is empty:

photoprism@220801-bookworm$ ./photoprism -o ./xxx index   
INFO[2022-08-01T15:00:27Z] indexing originals in ./xxx 
INFO[2022-08-01T15:00:27Z] originals folder is empty                    
INFO[2022-08-01T15:00:27Z] indexed 0 files in 543.707523ms              
INFO[2022-08-01T15:00:27Z] closed database connection 

Of course, that doesn't help if only part of your files are missing.

Please help test these changes while we are on vacation!

Thank you 👍

[lastzero]

Started a new preview build for testing! 👍

[aldobranti]

I have an interest in seeing this fixed -- I thought I could docker pull photoprism/photoprism:preview and test it from there, but obvs that's not the way it works.
I am running on arm64 how do I get something I can test without too much contortion?

[aldobranti]

I got hold of photoprism/photoprism:preview and ran it up
`PhotoPrism® CE
Build 220629-5d7448d2

turn on debug logs

docker restatrt

"offline originals drive"

trigger full rescan in gui

logs read as follows

time="2022-08-30T16:33:31Z" level=info msg="settings saved"
time="2022-08-30T16:33:31Z" level=debug msg="http: POST /api/v1/settings (200) [8.429127ms]"
time="2022-08-30T16:33:41Z" level=info msg="indexing originals..."
time="2022-08-30T16:33:41Z" level=info msg="classify: loading nasnet"
time="2022-08-30T16:33:52Z" level=info msg="classify: loading labels from labels.txt"
time="2022-08-30T16:33:52Z" level=info msg="index: found no .ppignore file"
time="2022-08-30T16:33:52Z" level=info msg="index: added folder /"
time="2022-08-30T16:33:52Z" level=info msg="index: found no new or modified files"
time="2022-08-30T16:33:53Z" level=debug msg="covers: updated 0 folders [776.71µs]"
time="2022-08-30T16:33:53Z" level=debug msg="removed folder:/:true:false from cache"
time="2022-08-30T16:33:53Z" level=debug msg="index: found no photos without primary file [738.34µs]"
time="2022-08-30T16:33:53Z" level=info msg="index: found no orphan files [559.083µs]"
time="2022-08-30T16:33:53Z" level=debug msg="search: updated file index [504.863µs]"
time="2022-08-30T16:33:53Z" level=debug msg="index: updating counts"
time="2022-08-30T16:33:53Z" level=debug msg="counts: updated 1 place [2.185595ms]"
time="2022-08-30T16:33:53Z" level=debug msg="counts: updated 0 subjects [1.613548ms]"
time="2022-08-30T16:33:53Z" level=debug msg="counts: updated 0 labels [1.006391ms]"
time="2022-08-30T16:33:53Z" level=debug msg="index: updating covers"
time="2022-08-30T16:33:53Z" level=debug msg="covers: updated 0 albums [1.314442ms]"
time="2022-08-30T16:33:53Z" level=debug msg="covers: updated 0 folders [826.154µs]"
time="2022-08-30T16:33:53Z" level=debug msg="api: abort /api/v1/config with code 401 (Please log in and try again)"
time="2022-08-30T16:33:53Z" level=debug msg="http: GET /api/v1/config (401) [306.458µs]"
time="2022-08-30T16:33:53Z" level=debug msg="covers: updated 0 months [2.383795ms]"
time="2022-08-30T16:33:53Z" level=debug msg="covers: updated 0 labels [3.638368ms]"
time="2022-08-30T16:33:53Z" level=debug msg="covers: updated 0 subjects [1.579623ms]"
time="2022-08-30T16:33:53Z" level=debug msg="moments: analyzing 0 photos and 0 videos, with threshold 3"
time="2022-08-30T16:33:53Z" level=debug msg="moments: not enough files"
time="2022-08-30T16:33:53Z" level=info msg="indexing completed in 11 s"
time="2022-08-30T16:33:53Z" level=debug msg="http: POST /api/v1/index (200) [11.898918282s]"
time="2022-08-30T16:33:53Z" level=debug msg="http: GET / (200) [7.946837ms]"
time="2022-08-30T16:33:53Z" level=debug msg="http: GET /api/v1/ws (200) [1m53.021085312s]"

"on line originals file system"
`
it wasn't quite enough, I had to restart the container for it to successfully run a complete index