Raw images are not getting scanned

[derneuere]

RAW images are not getting scanned on demo2. If you select the subfolder, it adds all the files to the scan, but then it will just skip them. This is either an ImageMagick issue or something that got introduced with live photos together.

[sickelap]

Could you provide a sample photo to reproduce the issue?

[derneuere]

librephotos.com/temp/testfiles.tar.gz

[sickelap]

I was unable to reproduce the issue. I mapped the whole archive of sample photos and selected only raw folder to be used as data directory for the admin user.

Pulled fresh docker images

reallibrephotos/librephotos-frontend   latest    437252fd10d9   3 days ago     25.2MB
reallibrephotos/librephotos            latest    f7d7b6450655   3 days ago     5.36GB
reallibrephotos/librephotos-proxy      latest    67b51f6e41d4   3 days ago     142MB
reallibrephotos/librephotos-base       dev       362341297280   3 weeks ago    4.23GB

[derneuere]

Thanks for looking into this :) @DigitalWarrior can you look into the server to maybe see what's up with the raw images? They are either missing or corrupted, I think

[DigitalWarrior]

Will get it fixed.

[DigitalWarrior]

Getting errors on latest and dev.
The Scan finds them and reports 5 photos. Only one photo gets a thumbnail generated though the jpeg.
During the creation of the thumbnails the other file formats error out. "reason: no decode delegate for this image format"

Testing with 5 photos.
jpeg - Processed correctly
CR2 - Error
CR3 - Error
NEF - Error

Please see attached logs. This was done with photos on the server not uploaded though the web.

Logs
https://linx.librephotos.com/ownphotos.log
https://linx.librephotos.com/commandmigrate.log
https://linx.librephotos.com/gunicorndjango.log
https://linx.librephotos.com/gunicornimagesimilarity.log
https://linx.librephotos.com/imagesimilarity.log
https://linx.librephotos.com/rqworker.log
https://linx.librephotos.com/showmigrate.log

Photos
https://linx.librephotos.com/signatureeditsfreerawsdsc4437.nef
https://linx.librephotos.com/monica1903.cr2
https://linx.librephotos.com/img6316.cr2
https://linx.librephotos.com/img1627.jpg
https://linx.librephotos.com/img1302.cr3

@derneuere can you dig into the thumbnail generation?

[derneuere]

The error is that ImageMagick can't find the correct deleagate, which is in this case libraw. We install libraw from source to get .cr3 support and that also used to work...

[derneuere]

Cant reproduce the issue at the moment, because I get a different error. ImageMagick and Wand show that they support cr3, for example, but when I scan the process it hangs and never finishes.

[sickelap]

Weird thing though is that the process hangs while generating the image here. Creating a test script to process any raw file within the backend container is successful.

root@27b92a8a2bec:/data# ls IMG_1811.*
IMG_1811.CR3
root@27b92a8a2bec:/data# cat test.py 
#!/usr/bin/env python
from wand.image import Image
with Image(filename="./IMG_1811.CR3") as img:
    with img.clone() as thumbnail:
        thumbnail.save(filename="./IMG_1811.webp")
root@27b92a8a2bec:/data# ./test.py
root@27b92a8a2bec:/data# ls IMG_1811.*
IMG_1811.CR3  IMG_1811.webp

[sickelap]

There are other issues related to managing model data, but that's not related to this one.

[derneuere]

It could be an issue like this one here, but I am not sure yet: emcconville/wand#536

[sickelap]

hah, interesting. I'll play around with it...

[sickelap]

#854 should fix this

[derneuere]

It's now fixed thanks to #854