HEIF: heif-convert must be updated to prevent empty JPEGs

[MoweME]

It would be awesome if someone could help us compile heif-convert with libjpeg80 as mentioned in this issue comment. If that fixes the problem, we don't have to spend time on a workaround! 💎💖

A bash script that we can drop into https://github.com/photoprism/photoprism/tree/develop/scripts/dist will do. It can then be run when building Docker images, but also by people who don't use Docker.

---

1. What is not working as documented?

Uploaded HEIC pictures can't be converted into JPEG.

2. How can we reproduce it?

Steps to reproduce the behavior:

1. Go into any album
2. Click on upload (top-right corner)
3. Upload HEIC picture
4. See error (Docker logs below)

Album is empty. Using current docker preview build.

3. What behavior do you expect?

PhotoPrism should create a JPEG version in addition to the original HEIC file.

4. What could be the cause of your problem?

Looks like the HEIC converter doesn't generate any output.

5. Can you provide us with example files for testing, error logs, or screenshots?

time="2022-09-21T22:41:23Z" level=debug msg="server: POST /api/v1/upload/1663800083546 (200) [47.557105ms]"
time="2022-09-21T22:41:23Z" level=debug msg="covers: updated 0 folders [537.901µs]"
time="2022-09-21T22:41:23Z" level=debug msg="removed folder:import:true:false from cache"
time="2022-09-21T22:41:23Z" level=info msg="moving files from 1663800083546"
time="2022-09-21T22:41:23Z" level=debug msg="import: adding files to album arikzl9131rb3gls"
time="2022-09-21T22:41:23Z" level=info msg="classify: loading nasnet"
time="2022-09-21T22:41:25Z" level=info msg="classify: loading labels from labels.txt"
time="2022-09-21T22:41:25Z" level=info msg="import: found no .ppignore file"
time="2022-09-21T22:41:25Z" level=info msg="import: added folder /upload/1663800083546"
time="2022-09-21T22:41:25Z" level=debug msg="exiftool: extracting metadata from upload/1663800083546/IMG_20201116_185837.HEIC"
time="2022-09-21T22:41:26Z" level=debug msg="import: created f7fa4a5305d9d9463d9bb7cd6621037450f57279_exiftool.json"
time="2022-09-21T22:41:26Z" level=info msg="media: IMG_20201116_185837.HEIC was taken at 2020-11-16 17:58:38 +0000 UTC (meta)"
time="2022-09-21T22:41:26Z" level=info msg="folder: added album 'November 2020' (path:2020/11 public:true)"
time="2022-09-21T22:41:26Z" level=info msg="import: created folder /2020/11"
time="2022-09-21T22:41:26Z" level=info msg="import: moving main heif file IMG_20201116_185837.HEIC to 2020/11/20201116_175838_21A8D266.heic"
time="2022-09-21T22:41:26Z" level=debug msg="failed renaming file, fallback to copy and delete: rename /photoprism/import/upload/1663800083546/IMG_20201116_185837.HEIC /photoprism/originals/2020/11/20201116_175838_21A8D266.heic: invalid cross-device link"
time="2022-09-21T22:41:26Z" level=info msg="convert: converting 20201116_175838_21A8D266.heic to 20201116_175838_21A8D266.heic.jpg (heif-convert)"
time="2022-09-21T22:41:26Z" level=error msg="import: Bogus marker length\ncould not write image\nWarning: File is empty - /photoprism/storage/sidecar/2020/11/20201116_175838_21A8D266.heic.jpg\nError: Format error in file - /photoprism/storage/sidecar/2020/11/20201116_175838_21A8D266.heic.jpg\n in 2020/11/20201116_175838_21A8D266.heic (convert to jpeg)"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no orphan markers [894.017µs]"
time="2022-09-21T22:41:26Z" level=debug msg="markers: found no invalid references [977.108µs]"
time="2022-09-21T22:41:26Z" level=debug msg="markers: found no missing subjects [314.67µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no ambiguous subjects [239.481µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no clusters to be merged [248.855µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no existing clusters"
time="2022-09-21T22:41:26Z" level=debug msg="faces: skipped clustering"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no new faces [480.439µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no unmatched markers"
time="2022-09-21T22:41:26Z" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [639.677µs]"
time="2022-09-21T22:41:26Z" level=debug msg="index: updating counts"
time="2022-09-21T22:41:26Z" level=debug msg="counts: updated 0 places [335.667µs]"
time="2022-09-21T22:41:26Z" level=debug msg="counts: updated 0 subjects [368.457µs]"
time="2022-09-21T22:41:26Z" level=debug msg="counts: updated 0 labels [325.041µs]"
time="2022-09-21T22:41:26Z" level=info msg="import: deleted empty folder /photoprism/import/upload/1663800083546"
time="2022-09-21T22:41:26Z" level=debug msg="moments: analyzing 0 photos and 0 videos, with threshold 3"
time="2022-09-21T22:41:26Z" level=debug msg="moments: not enough files"
time="2022-09-21T22:41:26Z" level=info msg="import completed in 3 s"
time="2022-09-21T22:41:26Z" level=debug msg="index: updating covers"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 albums [391.642µs]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 1 folder [2.99018ms]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 months [379.994µs]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 labels [435.439µs]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 subjects [293.05µs]"
time="2022-09-21T22:41:26Z" level=debug msg="server: POST /api/v1/import/upload/1663800083546 (200) [3.155598728s]"
time="2022-09-21T22:41:26Z" level=debug msg="server: GET /api/v1/albums?q=&count=1000&offset=0&type=album (200) [810.043µs]"
time="2022-09-21T22:41:26Z" level=debug msg="photos: found 0 results for dist:20 album:arikzl9131rb3gls camera:0 count:120 order:oldest merged:true [701.695µs]"

6. Which software versions do you use?

(a) Build 220919-cc8bab446 / AMD64

(b) Database Type & Version: mariadb:10.6

(c) Operating System Types & Versions: Docker (photoprism/photoprism:preview)

(d) Browser Types & Versions: Any browser tested. (Mainly tested Brave for Windows v1.43.93)

(e) Ad Blockers, Browser Plugins, and/or Firewall Software?: Everything disabled.

7. On what kind of device is PhotoPrism installed?

This is especially important if you are reporting a performance, import, or indexing issue. You can skip this if you're reporting a problem you found in our public demo, or if it's a completely unrelated issue, such as incorrect page layout.

(a) Device / Processor Type: 8 x Intel(R) Core(TM) i3-10100 CPU @ 3.60GHz (1 Socket)

(b) Physical Memory & Swap Space in GB:

(c) Storage Type: Containers installed on SSDs, NAS Storage mounted (original pictures)

(d) Anything else that might be helpful to know?: All types of images work except HEIC

[lastzero]

I've searched the error message on Google and found this issue (it's the tool we use unless you run PhotoPrism directly on a Mac, which would use Sips instead):

• heif-convert to jpeg failed for some HEIC-files => "Bogus marker length" error strukturag/libheif#575

[MoweME]

Looked up this issue by myself for hours and didn't saw that issue, thanks!
Considering that the issue has been open since December 12, 2021, is it an option to add a setting to select the output format of the conversion as a workaround?

[lastzero]

PNG files would be huge. In a perfect world, someone would help us compile heif-convert with libjpeg80 as described in the issue comment. If that fixes the problem, we don't have to spend time on a workaround! 💎💖

[eli103]

Did you upload HEIC with iPhone?

same issue.

Replicate:
To upload a same HEIC photo from IPhone and PC for test.
Upload from PC:(chrome) all good, it keep the original HEIC photo, as well as the photo metadata.
Upload from iPhone directly(safari/chrome), the HEIC original photo lost and only uploaded a converted jpeg file to server. And lost the photo metadata. ( tried use the browser’s desktop website mode, no luck)

[hiro5id]

Did you upload HEIC with iPhone?

same issue.

Replicate:

To upload a same HEIC photo from IPhone and PC for test.

Upload from PC:(chrome) all good, it keep the original HEIC photo, as well as the photo metadata.

Upload from iPhone directly(safari/chrome), the HEIC original photo lost and only uploaded a converted jpeg file to server. And lost the photo metadata. ( tried use the browser’s desktop website mode, no luck)

I wonder if this is a setting in iPhones safari, that it does the JPG conversion during upload.

[MoweME]

Did you upload HEIC with iPhone?
same issue.
Replicate:
To upload a same HEIC photo from IPhone and PC for test.
Upload from PC:(chrome) all good, it keep the original HEIC photo, as well as the photo metadata.
Upload from iPhone directly(safari/chrome), the HEIC original photo lost and only uploaded a converted jpeg file to server. And lost the photo metadata. ( tried use the browser’s desktop website mode, no luck)

I wonder if this is a setting in iPhones safari, that it does the JPG conversion during upload.

It isn't. I don't use iOS. All of my pictures were taken with a Xiaomi Mi 10 Ultra.

[srett]

Anyone got an affected sample image around? Tried with a random HEIF image from google and it worked with the stock heif-convert on Debian.

Also, there is some confusion about what libjpeg80 is even supposed to be... Pretty much every distro uses libjpeg-turbo instead of the reference implementation, and that one is only at library version 62 as far as I can tell. The reference implementation is at 9e (library version 9); I could get heif-convert to compile with it, and while doing so noticed that it doesn't have the jpeg_write_icc_profile function that -turbo has and which heif-convert uses. In that case, heif-convert uses its own implementation of that function. If we're lucky, it's exactly that function that causes issues with libjpeg-turbo, while the built-in version is fine, so in that case one could simply patch the build process to always use the built-in version, so no need to ship a different jpeg lib.

[lastzero]

@srett Thanks a lot for looking into this! I didn't know about libjpeg80 and libjpeg-turbo before, as Go has its own JPEG implementation. However, I just found an email from a Chinese user who reported this problem before and fixed it with his own heif-convert binary based on libjpeg80. Do we have your email so I can forward it?

[srett]

@lastzero Just dropped you a mail