Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Index: Rescan doesn't remove deleted folders #1010

Closed
maltokyo opened this issue Feb 6, 2021 · 61 comments
Closed

Index: Rescan doesn't remove deleted folders #1010

maltokyo opened this issue Feb 6, 2021 · 61 comments
Assignees
Labels
bug Something isn't working released Available in the stable release

Comments

@maltokyo
Copy link

maltokyo commented Feb 6, 2021

I have about 1000 folders (400k photos) in my originals directory, and I deleted (actually moved) about 200 of those to another drive which I don't want to be displayed in Photoprism.

The folders have been removed, but still show up in the gallery, including the ability to view them.

How can I remove these from Photoprism altogether? I realize a complete rescan may work, but the first scan took about two weeks! So I don't want to do that again..

Any advice?

@maltokyo maltokyo changed the title Folders deleted from Originals do not get deleted in album Folders deleted from Originals do not get deleted from Photoprism Feb 6, 2021
@lastzero
Copy link
Member

lastzero commented Feb 6, 2021

Regular rescan doesn't work?

@maltokyo
Copy link
Author

maltokyo commented Feb 6, 2021

Unfortunately not, I have tried it a few times, the old photos still there..

@lastzero
Copy link
Member

lastzero commented Feb 6, 2021

Also when you manually run photoprism purge in a terminal?

@maltokyo
Copy link
Author

maltokyo commented Feb 6, 2021

oh let me see how to do this, I am only using web GUI.

@maltokyo
Copy link
Author

maltokyo commented Feb 6, 2021

I am in the docs advanced section, but cannot find CLI guide, could you please help as to where that is?
https://docs.photoprism.org/user-guide/settings/advanced/

@lastzero
Copy link
Member

lastzero commented Feb 6, 2021

Getting Started as well as in our Docker example config file. Look for command reference.

@maltokyo
Copy link
Author

maltokyo commented Feb 6, 2021

ah, you just mean simply there is a command photoprism purge - sorry, I will try this.
Thank you for your help.

@maltokyo
Copy link
Author

maltokyo commented Feb 6, 2021

thank you, that finished as below, and says it deleted a bunch of things, but I can still see them in the gallery.

WARN[2021-02-06T19:55:07Z] photo: photo pqnd7992yynrln6m has no jpegs (set new primary) 
INFO[2021-02-06T19:55:07Z] purge: searching index for hidden media files 
INFO[2021-02-06T19:55:20Z] purge: removed 22913 files and 33500 photos in 8m12.047469537s 
INFO[2021-02-06T19:55:20Z] closed database connection  

That said, the machine still has a huge process running darktable-cli as below, so maybe it still needs time. Ill leave it running overnight and see.

top - 21:22:28 up  2:48,  0 users,  load average: 18.02, 13.15, 7.19
Tasks:   5 total,   2 running,   3 sleeping,   0 stopped,   0 zombie
%Cpu(s): 57.8 us, 18.5 sy,  0.0 ni, 15.5 id,  8.1 wa,  0.0 hi,  0.1 si,  0.0 st
MiB Mem :   7925.8 total,   5259.1 free,   2371.6 used,    295.1 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   5293.1 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                         
 4573 root      20   0  144016  53856  33484 R  12.6   0.7   0:00.38 darktable-cli 

@lastzero lastzero changed the title Folders deleted from Originals do not get deleted from Photoprism Index: Rescan doesn't remove deleted folders Feb 7, 2021
@lastzero lastzero self-assigned this Feb 7, 2021
@lastzero lastzero added the bug Something isn't working label Feb 7, 2021
@lastzero lastzero added this to Development 🐝 in Roadmap 🚀✨ Feb 7, 2021
@lastzero lastzero added the waiting Impediment / blocked / waiting label Feb 7, 2021
@lastzero
Copy link
Member

lastzero commented Feb 7, 2021

Let us know if this is a major bug we need to fix... otherwise we'll tag a new release tomorrow.

@maltokyo
Copy link
Author

maltokyo commented Feb 7, 2021

Yes, unfortunately, the photos (at least generated previews of photos) are still available, even though the folder containing them is no longer mounted in the docker at all (after docker restart).

I am mounting the originals folder on the host machine (debian10) as read-only, I am not sure if it makes a difference? But I thought a regular rescan would detect removed photos.

How can I give you the proper logs etc, to have a look at?

@lastzero
Copy link
Member

lastzero commented Feb 7, 2021

Do you see any errors in the log? You may also send your logs to hello@photoprism.app

@graciousgrey is going to test this again with our current preview build. It's going to be the next stable version (unless we have to fix this issue), so trying it should be safe.

@lastzero
Copy link
Member

lastzero commented Feb 7, 2021

Might be related to readonly mode, maybe because generated JPEGs don't get removed and stay in the index...

@lastzero
Copy link
Member

lastzero commented Feb 7, 2021

There was a similar issue recently: Purge: Can't remove deleted TIFF files #917

@maltokyo
Copy link
Author

maltokyo commented Feb 7, 2021

Where are the logs stored. I had a look in the storage mount, and cannot see. In the database maybe? Please let me know, and I will email to you.

In the meantime, in the storage docker volume, I can see as attached, in the sidecar folder, the folders with the red box are all completely removed in the originals directory. But still in the sidecar files..

image

@maltokyo
Copy link
Author

maltokyo commented Feb 7, 2021

sorry, i thought i put a red box around the folders. Basically, from _queenie folder down to and including the _WIP_q folder, these are all not existing in the originals folder.

@lastzero
Copy link
Member

lastzero commented Feb 7, 2021

Warnings & errors are here: https://demo.photoprism.org/library/errors

All logs, incl debug and info messages, are sent to Docker... up to you where / if to store them.

@maltokyo
Copy link
Author

maltokyo commented Feb 7, 2021

There are thousands of this error in library/errors

image

@maltokyo
Copy link
Author

maltokyo commented Feb 7, 2021

I have also sent the logs, as a zip file to your email above. Please keep those private, clearly. Thank you again for all of your help, and the great project!

@lastzero
Copy link
Member

lastzero commented Feb 7, 2021

Maybe also try the cleanup command...

@maltokyo
Copy link
Author

maltokyo commented Feb 7, 2021

trying cleanup now, its running

@maltokyo
Copy link
Author

maltokyo commented Feb 7, 2021

Still going, will let it run overnight, have a good evening, from Switzerland ;)

@lastzero
Copy link
Member

lastzero commented Feb 8, 2021

Just added a folder and removed it - works perfectly:

delete

Based on this, I've decided to reduced the log level for missing primary files to info as that's what you would expect if you delete them:

again

What database are you using? Maybe there's an issue with the query... didn't have time to look at your full logs yet.

@maltokyo
Copy link
Author

maltokyo commented Feb 8, 2021

I keep running cleanup and purge, multiple times successively. They keep removing entires / thumbnails etc each run.. different numbers each time.

@maltokyo
Copy link
Author

maltokyo commented Feb 8, 2021

Final update - yes it seems there is a bug:

Running this (as below, you get the picture...) finally cleaned everything up. Each one kept purging and cleaning files, decreasing in number each time. Seems it could be a good old binary search issue, that the search just skips items when there are too many.

photoprism purge && photoprism cleanup && photoprism purge && photoprism cleanup && photoprism purge && photoprism cleanup && photoprism purge && photoprism cleanup && photoprism purge && photoprism cleanup && photoprism purge && photoprism cleanup && photoprism purge && photoprism cleanup && photoprism purge && photoprism cleanup

After 15-20 cycles of the above, above command eventually gave this outcome consistently:

INFO[2021-02-08T12:06:54Z] purge: searching index for hidden media files 
INFO[2021-02-08T12:07:02Z] purge: removed 0 files and 0 photos in 1m5.9222824s 
INFO[2021-02-08T12:07:02Z] closed database connection                   
INFO[2021-02-08T12:07:05Z] cleanup: read-only mode enabled              
INFO[2021-02-08T12:07:10Z] cleanup: removed 0 index entries and 0 orphan thumbnails in 6.149076047s 
INFO[2021-02-08T12:07:10Z] closed database connection  

@lastzero
Copy link
Member

lastzero commented Feb 8, 2021

Purge already fetches files and photos in batches: https://github.com/photoprism/photoprism/blob/develop/internal/photoprism/purge.go

This query might be wrong though (looks like a duplicate WHERE):

func ResetPhotoQuality() error {

@lastzero
Copy link
Member

lastzero commented Feb 8, 2021

Started a new preview build so that you may test this if you like - although the issue seems already fixed solved?

https://drone.photoprism.app/photoprism/photoprism/1037

@lastzero lastzero added please-test Ready for acceptance test and removed waiting Impediment / blocked / waiting labels Feb 8, 2021
@lastzero lastzero moved this from Development 🐝 to Preview 🐳 in Roadmap 🚀✨ Feb 8, 2021
@lastzero
Copy link
Member

lastzero commented Feb 8, 2021

Tested on my own (large) photo library: Found 3 additional photos to delete 👍

lastzero added a commit that referenced this issue Feb 8, 2021
@lastzero
Copy link
Member

lastzero commented Feb 8, 2021

Folders with hidden or missing photos only should be hidden now as well.

@maltokyo
Copy link
Author

Tested now, same result. But I already had zero results, so I cannot confirm if it is fixed, but indeed things are solved for now for me. Let me "delete" another folder, and try it out. Will come back with more results soon.

@maltokyo
Copy link
Author

Yes, it did find more, and after one purge and one cleanup they were all gone after one run of each command. Seems this has fixed it! Thank you!

@lastzero
Copy link
Member

Excellent, thanks for testing!

@lastzero lastzero added released Available in the stable release and removed please-test Ready for acceptance test labels Feb 10, 2021
@graciousgrey graciousgrey moved this from Preview 🐳 to Released 🌈 in Roadmap 🚀✨ Feb 11, 2021
@kevinmkr
Copy link

I just had this same issue while running the latest version of PhotoPrism through Docker on an UnRAID server, I'm sorry to say.

MariaDB.

I had removed 66 folders consisting of 46,000 photos from my physical location.

Executing the "purge/cleanup/purge/cleanup/etc..." routine a few times eventually cleared it all out.

I'm not sure if there's anything that I can offer in terms of logs, now that it's cleaned up. But, I can definitely suggest that, based on my experience, running purge/cleanup a single time was insufficient under these conditions.

@lastzero
Copy link
Member

lastzero commented Jan 14, 2022

How long did you wait? Could be a caching issue that eventually resolves itself after indexing & waiting a little.

Also possible that related sidecar files were left. In that case the solution is to delete them as well, which is what happens if you use the UI to delete. Might have to implement a delete sidecar command for manually deleted originals.

@kevinmkr
Copy link

kevinmkr commented Jan 14, 2022

I'm a bit untrustsworthy on my memory of the timeline, to be honest, so I'm not sure if it's even of any value to speculate as to how long I waited. It could very well have simply been the issue as you've described it with the caching.

Although, I can say that every time I executed the purge/cleanup, it DID find items (for the first few dozen iterations of those commands). So, would that still point towards the cache? Seems like it wouldn't.

@kevinmkr
Copy link

kevinmkr commented Jan 14, 2022

I can offer further information (Sorry -- missed that part in my initial read of your response):

The removal of the files occurred outsite of the UI. I physically removed them from the file system instead of "Deleting" them from the UI.
The purge/cleanup cycles occurred in the terminal.
I DID run the re-index command from the UI a couple of times, including with the "Complete Rescan" checked. But, again, the order of when I did what and the time in between the operations has been "purged" from my brains, unfortunately.

@lastzero
Copy link
Member

Rather indicates a problem with orphaned sidecar files... Folders that no longer exist should be hidden / deleted. However, if sidecar entries still exist in the index, this is exactly what can prevent this from functioning.

@kevinmkr
Copy link

Was there an additional command that I should have executed to axe the sidecar orphans?

@lastzero
Copy link
Member

We have to focus on completing multi-user features at the moment and work overtime to provide support to as many users as possible. Can't tell you more without digging deeper into details.

The code is public, so in theory others could take a look as well... 😉

@lastzero
Copy link
Member

  • Needless to say, we tested it all before releasing it, so it should generally work. It could be an edge case, 46,000 photos is a lot and no folder structure is the same.
  • Note that there was a bug in a previous version that prevented certain index entries from being permanently removed when you delete a photo.
  • So the behaviour depends not only on the version you are currently using, but also on when you started deleting photos.

@kevinmkr
Copy link

kevinmkr commented Jan 14, 2022

No worries. I wasn't clear on how to locate the version -- UnRAID simply says "up to date" and I didn't see anything within the UI. Running "photoprism version" from the terminal gave: "211018-e200f322-Linux-x86_64". Looking at the release notes, this looks like October 18, 2021. So, this could be the problem.

Still cutting my teeth on the UnRaid experience and even moreso on Docker.

Sorry for what was likely a false alarm based on an outdated build.

@lastzero
Copy link
Member

In this case, your issue may already be solved 💎

@kevinmkr
Copy link

:)

@ChenSun-Phys
Copy link

Running purge multiple times solved it. Although a bit tangential to the original issue -- would there be a way to help with the documentation where the explanations of purge and cleanup are currently missing?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working released Available in the stable release
Projects
Roadmap 🚀✨
Released 🌈
Development

No branches or pull requests

5 participants