Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import: Never remove ignored folders such as for Syncthing #1319

Closed
AudunVN opened this issue May 22, 2021 · 17 comments
Closed

Import: Never remove ignored folders such as for Syncthing #1319

AudunVN opened this issue May 22, 2021 · 17 comments
Assignees
Labels
enhancement Refactoring, improvement or maintenance task released Available in the stable release

Comments

@AudunVN
Copy link
Sponsor

AudunVN commented May 22, 2021

I'm like #280 using Syncthing to put files in my import directory from various devices. Syncthing creates and requires an empty folder named .stfolder to mark its folders, which is currently deleted by Photoprism (which subsequently breaks Syncthing) after an import even if .stfolder is specified in the .ppignore file in the import folder.

if subPath != "" && path != conf.ImportPath() && fs.IsEmpty(path) {
if err := os.Remove(path); err != nil {
log.Errorf("import: failed deleting empty folder %s: %s", txt.Quote(path), err)
} else {
log.Infof("import: deleted empty folder %s", txt.Quote(path))
}
}

Putting a file such as ppkeep in .stfolder does prevent deletion after import without affecting Syncthing, so it's not a large issue for my use case at least.

I'm not sure if this is a bug or feature request since the documentation I've found doesn't specify that Photoprism won't delete folders specified in .ppignore, but I at least initially expected .ppignore would protect the folder from being deleted from previous .ignore-type files I've used.

@AudunVN AudunVN changed the title Photoprism empty folder deletion during import does not use .ppignore Empty folder deletion during import does not use .ppignore May 22, 2021
@lastzero
Copy link
Member

lastzero commented May 23, 2021

When you chose not to move files from import but to copy them only - this way, nothing gets deleted. If you selected "move" in Library > Import, then a .ppignore file still won't prevent empty folders in import from getting deleted.

@lastzero
Copy link
Member

lastzero commented May 23, 2021

Maybe we can add a list of protected folders like for Syncthing and Git that never get touched. Would that help?

.stfolder
.git

@lastzero lastzero changed the title Empty folder deletion during import does not use .ppignore Import: Don't remove protected folders such as for Syncthing May 23, 2021
@lastzero lastzero self-assigned this May 23, 2021
@lastzero lastzero added the enhancement Refactoring, improvement or maintenance task label May 23, 2021
@AudunVN
Copy link
Sponsor Author

AudunVN commented May 23, 2021

That should help a lot!

I specifically use move instead of copy since I want Syncthing to automatically remove the imported files from my local Photoprism import folders once the import is done. For example, I have Syncthing set up to sync the DCIM folder on my phone with a folder in Photoprism's import directory on my server, so by using move on import I can free up space on my phone.

@lastzero
Copy link
Member

How does Syncthing know the import is done?

@AudunVN
Copy link
Sponsor Author

AudunVN commented May 23, 2021

Ah, sorry, I worded that a bit unclearly; Syncthing doesn't wait for the import to finish, so it doesn't need to know when the import is done. All it does is keep folders on different devices synchronized, so if Photoprism removes a file from a folder watched by Syncthing during an import these will also be removed from any synchronized devices.

My current setup is more or less

  • /photoprism/originals synchronized with a folder on my backup drives
  • Phone DCIM folder synchronized with /photoprism/import/phone
  • Laptop import folder synchronized with /photoprism/import/laptop
  • Desktop import folder synchronized with /photoprism/import/desktop

where I usually regularly copy the DCIM folders from my cameras to the laptop/desktop import folders. Photoprism is running on a hosted remote VPS.

This lets me back up and import images independently of the SD card or camera the images came from as soon as they've been copied to my laptop or desktop import folder. Works quite well when offline or on a slow network connection where uploading directly from the SD card or camera to Photoprism through WebDAV or similar would have issues.

@lastzero lastzero changed the title Import: Don't remove protected folders such as for Syncthing Import: Never remove ignored folders such as for Syncthing May 23, 2021
@lastzero
Copy link
Member

Started a preview build for testing my changes: https://drone.photoprism.app/photoprism/photoprism/

@lastzero lastzero added the please-test Ready for acceptance test label May 23, 2021
@AudunVN
Copy link
Sponsor Author

AudunVN commented May 23, 2021

Nice, thank you very much! 😃 The changes in 65c3967 seem like they should good solution for this indeed, will see if I can test it soon.

@lastzero
Copy link
Member

We decided to remove such folders as some operating systems always create hidden files and folders for keeping extended attributes, so practically empty directories would never get deleted. Some less experienced users might now wonder why their empty folders are still around - that's the tradeoff.

@lastzero
Copy link
Member

Updated Docker images are available now.

@lastzero
Copy link
Member

We've released it already as it works for us, just tested it again. Let us know if you still have issues.

@lastzero lastzero added released Available in the stable release and removed please-test Ready for acceptance test labels May 23, 2021
@AudunVN
Copy link
Sponsor Author

AudunVN commented May 23, 2021

Thank you very much again - will do!

@AudunVN
Copy link
Sponsor Author

AudunVN commented May 30, 2021

Ah, sorry, I worded that a bit unclearly; Syncthing doesn't wait for the import to finish, so it doesn't need to know when the import is done. All it does is keep folders on different devices synchronized, so if Photoprism removes a file from a folder watched by Syncthing during an import these will also be removed from any synchronized devices.

My current setup is more or less

  • /photoprism/originals synchronized with a folder on my backup drives
  • Phone DCIM folder synchronized with /photoprism/import/phone
  • Laptop import folder synchronized with /photoprism/import/laptop
  • Desktop import folder synchronized with /photoprism/import/desktop

where I usually regularly copy the DCIM folders from my cameras to the laptop/desktop import folders. Photoprism is running on a hosted remote VPS.

This lets me back up and import images independently of the SD card or camera the images came from as soon as they've been copied to my laptop or desktop import folder. Works quite well when offline or on a slow network connection where uploading directly from the SD card or camera to Photoprism through WebDAV or similar would have issues.

Just wanted to leave a quick note to verify that updating Photoprism to build 210523-b1856b9d and doing an import works perfectly with this setup! 😄

@HaroldDLeon
Copy link

Maybe we can add a list of protected folders like for Syncthing and Git that never get touched. Would that help?

.stfolder
.git

Would it be possible to also include .stignore on this allow-list?
I'm using a setup pretty similar to the above, but .stignore gets deleted on import.

@lastzero
Copy link
Member

lastzero commented Jun 3, 2021

Are you using our latest version? .folders should all be ignored now

@HaroldDLeon
Copy link

I am using the latest version. .stignore is a file (not sure if it'd be treated different than a .folder).

@lastzero
Copy link
Member

lastzero commented Jun 3, 2021

Add it to a ".ppignore" file then

@HaroldDLeon
Copy link

Following the discussion on #1348

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Refactoring, improvement or maintenance task released Available in the stable release
Projects
Status: Release 🌈
Development

No branches or pull requests

3 participants