Import: Syncthing's .stignore is deleted on import

[HaroldDLeon]

Follow-up discussion from #1319 :

When importing, Photoprism will delete the .stignore file from the folder, even if it's included on .ppignore.

[AudunVN]

Can reproduce this on build 210523-b1856b9d, looks like dotfiles (besides .ppignore) are quietly deleted even if included in .ppignore. Below is a copy of the log from an import where the files phone/.stignore, laptop/.stignore, drive/.stignore and desktop/.stignore were deleted.

2021-06-03 22:00:26 INFO Import completed in 7 s
2021-06-03 22:00:19 INFO import: ignored "phone/folder.ico"
2021-06-03 22:00:19 INFO import: ignored "phone/desktop.ini"
2021-06-03 22:00:19 INFO import: ignored "phone/OpenCamera/ppkeep"
2021-06-03 22:00:19 INFO import: ignored "phone/100ANDRO/ppkeep"
2021-06-03 22:00:19 INFO import: ignored "laptop/folder.ico"
2021-06-03 22:00:19 INFO import: ignored "laptop/desktop.ini"
2021-06-03 22:00:19 INFO import: ignored "drive/folder.ico"
2021-06-03 22:00:19 INFO import: ignored "drive/desktop.ini"
2021-06-03 22:00:19 INFO import: ignored "desktop/folder.ico"
2021-06-03 22:00:19 INFO import: ignored "desktop/desktop.ini"
2021-06-03 22:00:19 INFO Moving files from import

My .ppignore files contain

# ignore syncthing folders and files
.stfolder
.stversions
.stignore

# ignore folder metadata
desktop.ini
folder.ico

# others
ppkeep

and seem to be working fine except for .stignore being deleted.

In my case I have .stignore configured precisely since I don't want the ignored desktop.ini and folder.ico files mentioned in the import log to be there in the first place, as they're not much use on the headless Linux-based server I'm running Photoprism on.

[alexislefebvre]

I think .ppignore is only used when indexing directories, not importing them. Maybe this part of the doc should be updated: https://docs.photoprism.org/user-guide/library/indexing/#ignoring-directories-and-files

[lastzero]

Should also work in import, maybe it's on the wrong directory level. If you import from a sub folder, the patent folders might not be checked. Can also be a bug, didn't have time to look at this in detail.

[AudunVN]

I think .ppignore not having an effect may not be the most important thing here. Dotfiles (and/or hidden files) being quietly deleted would come as a surprise after reading the documentation, as one of the listed advantages of importing instead of indexing is

Unsupported files stay untouched in the import directory

which would likely make users expect that the unsupported file .stignore shouldn't be affected at all, regardless of whether a .ppignore file is set up. Not sure if this is a documentation or code issue, though.

PS: Thank you very much for all the help! 😀 Sorry for the Syncthing-related issues recently, even if I'm slightly glad it's not just me that's using Photoprism and (breaking) Syncthing in weird ways.

[lastzero]

You're welcome to add a note to our docs. Most modern operating systems use hidden files to store folder metadata, so in practice deleting empty folder would never work and would likewise lead to complaints, and bug reports.

[AudunVN]

Yep, I very much understand - would be a bit confusing if seemingly empty folders were left behind on import if the user doesn't know about or can't see the hidden files. .ppignore not working as expected is indeed the main concern here, then, since deleting hidden files is the intended behaviour. I'll see if I can add a note to the docs about all hidden files being deleted on import.

[HaroldDLeon]

Should also work in import, maybe it's on the wrong directory level. If you import from a sub folder, the patent folders might not be checked. Can also be a bug, didn't have time to look at this in detail.

My folder structure looks like this:

Imports/
├─ DCIM/
│  ├─ .ppignore
│  ├─ .stignore
│  ├─ .stfolder/
│  ├─ .stversions/
├─ .ppignore
├─ .stignore

With .ppignore having only one line, which is ignored:

.stignore

Also tested adding other dotfiles to the folder and to .ppignore, but they were also deleted on Import.

[lastzero]

Did you try DCIM/.stignore?

[HaroldDLeon]

Yes. On import, the folder structure from above comment would change to:

Imports/
├─ DCIM/
│  ├─ .ppignore
│  ├─ .stfolder/
│  ├─ .stversions/
├─ .ppignore

[lastzero]

We can have a look after our vacation, in about two weeks. It's probably easy to fix.