delete_missing_photos crashes after 30sec or so

[linuxturtle]

I'm running the "latest" docker image on a debian amd64 VPS, and I've started with a fairly large photo archive (97575 photos). I recently went through the archive on the filesystem, and deduplicated and restructured the archive, so I now have 81116 photos on the filesystem. So obviously, when I re-scan the archive, I get a lot of missing photos (14014, according to the library page, even though those numbers don't quite add up). Anyway, when I click the "Remove missing photos" button, the job starts running, then about 30sec in, I see this traceback in the ownphotos.log, and of course, the database never gets cleaned up:

2022-10-29 03:36:13,583 : views.py : get : 223 : ERROR : An Error occurred
Traceback (most recent call last):
  File "/code/api/views/views.py", line 220, in get
    delete_missing_photos(request.user, job_id)
  File "/code/api/autoalbum.py", line 218, in delete_missing_photos
    album_thing.photos.remove(missing_photo)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/fields/related_descriptors.py", line 967, in remove
    self._remove_items(self.source_field_name, self.target_field_name, *objs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/fields/related_descriptors.py", line 1183, in _remove_items
    with transaction.atomic(using=db, savepoint=False):
  File "/usr/local/lib/python3.10/dist-packages/django/db/transaction.py", line 232, in __exit__
    connection.commit()
  File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 266, in commit
    self._commit()
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 242, in _commit
    return self.connection.commit()
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/workers/base.py", line 203, in handle_abort
    sys.exit(1)
SystemExit: 1

The photo archive {scanDirectory} itself is NFS-mounted on the VPS, and the database, thumbnails, logs, and everything else is on a local SSD. Everything else is a pretty bog-standard docker-compose install.

[linuxturtle]

@loulou91 on discord is much smarter than me, and easily reproduced this in message https://discord.com/channels/784619049208250388/784619049208250395/1038165724230844456

"it is easy to reproduce with a test database. To produce quickly a large number of small pictures, I use faces from my ops base and I create a test database from scratch with these files (1415 photos in my case). They are quickly scanned, of course! Then I manually remove all photos from /data folder. I start a new scan and the Library tab tells me 1415 missing photos. Then and last "remove missing photos" fails and attached is the log."

[mihajlov123]

Having same problems as @linuxturtle. Is there any workarounds (besides fresh install) ?

[WeeBo-X]

Having the same issue as well, would like to resolve instead of starting from scratch.

[dfduarte78]

Im stuck too, its been 24hr and still going even after reboot the server. Does it ever ends?

[linuxturtle]

It's not still going, the process crashes, but the web UI doesn't register that, so it shows that it's still running forever.

[2pacsawyer]

I am also having the same issue with a similar log error. After I deleted photos from the local file system. I ran a scan photos and a rescan all photos job in Librephotos. After I run the remove missing photos job it errors out.

2023-03-29 16:05:19,879 : views.py : get : 218 : ERROR : An Error occurred Traceback (most recent call last): File "/code/api/views/views.py", line 215, in get delete_missing_photos(request.user, job_id) File "/code/api/autoalbum.py", line 223, in delete_missing_photos

I'm running on the Dev branch as of Monday 3/27
Docker install hosted on Ubuntu
started with 26953 photos
Librephotos now says I have 7558 missing photos

[Nerodacles]

Mine just stuck, it doesn't error out.

#774

[2pacsawyer]

Mine just stuck, it doesn't error out.

#774

@Nerodacles It looks like it's still running or stuck on the web interface but if you check /data/logs/ownphotos.log you might find an error indicating that it failed and is actually not running anymore.

[Nerodacles]

Mine just stuck, it doesn't error out.
#774

@Nerodacles It looks like it's still running or stuck on the web interface but if you check /data/logs/ownphotos.log you might find an error indicating that it failed and is actually not running anymore.

Noup, it says finished in the log, but doesn't error out!

[Nerodacles]

With the latest dev branch it is completed, and not a bug.