Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HEIF: heif-convert must be updated to prevent empty JPEGs #2726

Closed
MoweME opened this issue Sep 21, 2022 · 46 comments
Closed

HEIF: heif-convert must be updated to prevent empty JPEGs #2726

MoweME opened this issue Sep 21, 2022 · 46 comments
Assignees
Labels
bug Something isn't working docker Docker Images, Build Scripts, Config & Deployment Examples released Available in the stable release

Comments

@MoweME
Copy link

MoweME commented Sep 21, 2022

It would be awesome if someone could help us compile heif-convert with libjpeg80 as mentioned in this issue comment. If that fixes the problem, we don't have to spend time on a workaround! 💎💖

A bash script that we can drop into https://github.com/photoprism/photoprism/tree/develop/scripts/dist will do. It can then be run when building Docker images, but also by people who don't use Docker.


1. What is not working as documented?

Uploaded HEIC pictures can't be converted into JPEG.

2. How can we reproduce it?

Steps to reproduce the behavior:

  1. Go into any album
  2. Click on upload (top-right corner)
  3. Upload HEIC picture
  4. See error (Docker logs below)

Album is empty. Using current docker preview build.

3. What behavior do you expect?

PhotoPrism should create a JPEG version in addition to the original HEIC file.

4. What could be the cause of your problem?

Looks like the HEIC converter doesn't generate any output.

5. Can you provide us with example files for testing, error logs, or screenshots?

time="2022-09-21T22:41:23Z" level=debug msg="server: POST /api/v1/upload/1663800083546 (200) [47.557105ms]"
time="2022-09-21T22:41:23Z" level=debug msg="covers: updated 0 folders [537.901µs]"
time="2022-09-21T22:41:23Z" level=debug msg="removed folder:import:true:false from cache"
time="2022-09-21T22:41:23Z" level=info msg="moving files from 1663800083546"
time="2022-09-21T22:41:23Z" level=debug msg="import: adding files to album arikzl9131rb3gls"
time="2022-09-21T22:41:23Z" level=info msg="classify: loading nasnet"
time="2022-09-21T22:41:25Z" level=info msg="classify: loading labels from labels.txt"
time="2022-09-21T22:41:25Z" level=info msg="import: found no .ppignore file"
time="2022-09-21T22:41:25Z" level=info msg="import: added folder /upload/1663800083546"
time="2022-09-21T22:41:25Z" level=debug msg="exiftool: extracting metadata from upload/1663800083546/IMG_20201116_185837.HEIC"
time="2022-09-21T22:41:26Z" level=debug msg="import: created f7fa4a5305d9d9463d9bb7cd6621037450f57279_exiftool.json"
time="2022-09-21T22:41:26Z" level=info msg="media: IMG_20201116_185837.HEIC was taken at 2020-11-16 17:58:38 +0000 UTC (meta)"
time="2022-09-21T22:41:26Z" level=info msg="folder: added album 'November 2020' (path:2020/11 public:true)"
time="2022-09-21T22:41:26Z" level=info msg="import: created folder /2020/11"
time="2022-09-21T22:41:26Z" level=info msg="import: moving main heif file IMG_20201116_185837.HEIC to 2020/11/20201116_175838_21A8D266.heic"
time="2022-09-21T22:41:26Z" level=debug msg="failed renaming file, fallback to copy and delete: rename /photoprism/import/upload/1663800083546/IMG_20201116_185837.HEIC /photoprism/originals/2020/11/20201116_175838_21A8D266.heic: invalid cross-device link"
time="2022-09-21T22:41:26Z" level=info msg="convert: converting 20201116_175838_21A8D266.heic to 20201116_175838_21A8D266.heic.jpg (heif-convert)"
time="2022-09-21T22:41:26Z" level=error msg="import: Bogus marker length\ncould not write image\nWarning: File is empty - /photoprism/storage/sidecar/2020/11/20201116_175838_21A8D266.heic.jpg\nError: Format error in file - /photoprism/storage/sidecar/2020/11/20201116_175838_21A8D266.heic.jpg\n in 2020/11/20201116_175838_21A8D266.heic (convert to jpeg)"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no orphan markers [894.017µs]"
time="2022-09-21T22:41:26Z" level=debug msg="markers: found no invalid references [977.108µs]"
time="2022-09-21T22:41:26Z" level=debug msg="markers: found no missing subjects [314.67µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no ambiguous subjects [239.481µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no clusters to be merged [248.855µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no existing clusters"
time="2022-09-21T22:41:26Z" level=debug msg="faces: skipped clustering"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no new faces [480.439µs]"
time="2022-09-21T22:41:26Z" level=debug msg="faces: found no unmatched markers"
time="2022-09-21T22:41:26Z" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [639.677µs]"
time="2022-09-21T22:41:26Z" level=debug msg="index: updating counts"
time="2022-09-21T22:41:26Z" level=debug msg="counts: updated 0 places [335.667µs]"
time="2022-09-21T22:41:26Z" level=debug msg="counts: updated 0 subjects [368.457µs]"
time="2022-09-21T22:41:26Z" level=debug msg="counts: updated 0 labels [325.041µs]"
time="2022-09-21T22:41:26Z" level=info msg="import: deleted empty folder /photoprism/import/upload/1663800083546"
time="2022-09-21T22:41:26Z" level=debug msg="moments: analyzing 0 photos and 0 videos, with threshold 3"
time="2022-09-21T22:41:26Z" level=debug msg="moments: not enough files"
time="2022-09-21T22:41:26Z" level=info msg="import completed in 3 s"
time="2022-09-21T22:41:26Z" level=debug msg="index: updating covers"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 albums [391.642µs]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 1 folder [2.99018ms]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 months [379.994µs]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 labels [435.439µs]"
time="2022-09-21T22:41:26Z" level=debug msg="covers: updated 0 subjects [293.05µs]"
time="2022-09-21T22:41:26Z" level=debug msg="server: POST /api/v1/import/upload/1663800083546 (200) [3.155598728s]"
time="2022-09-21T22:41:26Z" level=debug msg="server: GET /api/v1/albums?q=&count=1000&offset=0&type=album (200) [810.043µs]"
time="2022-09-21T22:41:26Z" level=debug msg="photos: found 0 results for dist:20 album:arikzl9131rb3gls camera:0 count:120 order:oldest merged:true [701.695µs]"

6. Which software versions do you use?

(a) Build 220919-cc8bab446 / AMD64

(b) Database Type & Version: mariadb:10.6

(c) Operating System Types & Versions: Docker (photoprism/photoprism:preview)

(d) Browser Types & Versions: Any browser tested. (Mainly tested Brave for Windows v1.43.93)

(e) Ad Blockers, Browser Plugins, and/or Firewall Software?: Everything disabled.

7. On what kind of device is PhotoPrism installed?

This is especially important if you are reporting a performance, import, or indexing issue. You can skip this if you're reporting a problem you found in our public demo, or if it's a completely unrelated issue, such as incorrect page layout.

(a) Device / Processor Type: 8 x Intel(R) Core(TM) i3-10100 CPU @ 3.60GHz (1 Socket)

(b) Physical Memory & Swap Space in GB:
image

(c) Storage Type: Containers installed on SSDs, NAS Storage mounted (original pictures)

(d) Anything else that might be helpful to know?: All types of images work except HEIC

@MoweME MoweME added the bug Something isn't working label Sep 21, 2022
@lastzero
Copy link
Member

lastzero commented Sep 22, 2022

I've searched the error message on Google and found this issue (it's the tool we use unless you run PhotoPrism directly on a Mac, which would use Sips instead):

@MoweME
Copy link
Author

MoweME commented Sep 22, 2022

Looked up this issue by myself for hours and didn't saw that issue, thanks!
Considering that the issue has been open since December 12, 2021, is it an option to add a setting to select the output format of the conversion as a workaround?

@lastzero
Copy link
Member

PNG files would be huge. In a perfect world, someone would help us compile heif-convert with libjpeg80 as described in the issue comment. If that fixes the problem, we don't have to spend time on a workaround! 💎💖

@lastzero lastzero added help wanted Well suited for external contributors! docker Docker Images, Build Scripts, Config & Deployment Examples labels Sep 22, 2022
@eli103
Copy link

eli103 commented Sep 27, 2022

Did you upload HEIC with iPhone?

same issue.

Replicate:
To upload a same HEIC photo from IPhone and PC for test.
Upload from PC:(chrome) all good, it keep the original HEIC photo, as well as the photo metadata.
Upload from iPhone directly(safari/chrome), the HEIC original photo lost and only uploaded a converted jpeg file to server. And lost the photo metadata. ( tried use the browser’s desktop website mode, no luck)

@hiro5id
Copy link

hiro5id commented Sep 28, 2022

Did you upload HEIC with iPhone?

same issue.

Replicate:

To upload a same HEIC photo from IPhone and PC for test.

Upload from PC:(chrome) all good, it keep the original HEIC photo, as well as the photo metadata.

Upload from iPhone directly(safari/chrome), the HEIC original photo lost and only uploaded a converted jpeg file to server. And lost the photo metadata. ( tried use the browser’s desktop website mode, no luck)

I wonder if this is a setting in iPhones safari, that it does the JPG conversion during upload.

@lastzero lastzero changed the title Bug: HEIC conversion creates empty JPEG HEIF: heif-convert must be updated to prevent empty JPEGs from being created Oct 1, 2022
@lastzero lastzero changed the title HEIF: heif-convert must be updated to prevent empty JPEGs from being created HEIF: heif-convert must be updated to prevent empty JPEGs Oct 1, 2022
@MoweME
Copy link
Author

MoweME commented Oct 1, 2022

Did you upload HEIC with iPhone?
same issue.
Replicate:
To upload a same HEIC photo from IPhone and PC for test.
Upload from PC:(chrome) all good, it keep the original HEIC photo, as well as the photo metadata.
Upload from iPhone directly(safari/chrome), the HEIC original photo lost and only uploaded a converted jpeg file to server. And lost the photo metadata. ( tried use the browser’s desktop website mode, no luck)

I wonder if this is a setting in iPhones safari, that it does the JPG conversion during upload.

It isn't. I don't use iOS. All of my pictures were taken with a Xiaomi Mi 10 Ultra.

@srett
Copy link
Contributor

srett commented Oct 2, 2022

Anyone got an affected sample image around? Tried with a random HEIF image from google and it worked with the stock heif-convert on Debian.

Also, there is some confusion about what libjpeg80 is even supposed to be... Pretty much every distro uses libjpeg-turbo instead of the reference implementation, and that one is only at library version 62 as far as I can tell. The reference implementation is at 9e (library version 9); I could get heif-convert to compile with it, and while doing so noticed that it doesn't have the jpeg_write_icc_profile function that -turbo has and which heif-convert uses. In that case, heif-convert uses its own implementation of that function. If we're lucky, it's exactly that function that causes issues with libjpeg-turbo, while the built-in version is fine, so in that case one could simply patch the build process to always use the built-in version, so no need to ship a different jpeg lib.

@lastzero
Copy link
Member

lastzero commented Oct 3, 2022

@srett Thanks a lot for looking into this! I didn't know about libjpeg80 and libjpeg-turbo before, as Go has its own JPEG implementation. However, I just found an email from a Chinese user who reported this problem before and fixed it with his own heif-convert binary based on libjpeg80. Do we have your email so I can forward it?

@srett
Copy link
Contributor

srett commented Oct 3, 2022

@lastzero Just dropped you a mail

lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
@lastzero
Copy link
Member

lastzero commented Oct 6, 2022

@srett Got it! Unfortunately there was no HEIF example attached to the email, so I can't share anything but the information I already shared (use libjpeg8).

I guess it's best to try libjpeg8, because a bad color profile is still better than no JPEG at all.... my build script is a mess and I don't have time to build for all the other Docker images. You are welcome to take care of it later if you want! :)

lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
@lastzero
Copy link
Member

lastzero commented Oct 6, 2022

The naming scheme for the downloadable binary archives could also use some love...

lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
@lastzero
Copy link
Member

lastzero commented Oct 6, 2022

Seems like the convert script isn't needed anymore with the new version of heif-convert!

lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
lastzero added a commit that referenced this issue Oct 6, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
@srett
Copy link
Contributor

srett commented Oct 7, 2022

Ah right, I only tested on Debian Bookworm, and it doesn't even ship libjpeg8, so I had to compile that from source too.

I guess it's best to try libjpeg8, because a bad color profile is still better than no JPEG at all....

If the jpeg_write_icc_profile implementation that comes with heif-convert isn't broken or severely outdated, it should have a proper color profile with that current solution :-)

But maybe @MoweME can provide an affected .heif file, so I can check what the options are. If there is a solution to this while keeping libjpeg62-turbo instead of libjpeg8 it would have the benefit of also working on Debian out of the box, should you decide to switch to that base image again.

@lastzero
Copy link
Member

lastzero commented Oct 7, 2022

ICC profiles are there, I just checked. It would be awesome if we could get more HEIF/HEIC examples for testing, especially if we can make them available on https://dl.photoprism.app/samples/!

lastzero added a commit that referenced this issue Oct 7, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
lastzero added a commit that referenced this issue Oct 7, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
@lastzero
Copy link
Member

Not at this time, because it adds complexity and will likely slow down indexing significantly as well. We can go that extra mile when there are fewer open issues and we have a support team that can help with any problems that might result.

@lastzero
Copy link
Member

To give a specific example: I'm not sure to what extent PNG supports the metadata of the regular JPEG file and possibly something might get lost when converting. It is possible to test and figure this out, but we don't have the time.

@lastzero
Copy link
Member

@larsxschneider As a workaround, you can simply change the name and/or path of the "heif-convert" binary in the config options so that you can continue to use your existing script.

@larsxschneider
Copy link

I will do that. Thank you for your explanations and the clear communication 🙇

@inthreedee
Copy link

@lastzero I think this is the right place to put this. I was indexing some new photos on the latest preview release Build 221025-3b9890e34 and a bunch of HEIC images were not converted to jpgs. They were all taken on the same phone, so I don't know why most indexed fine but others did not. In the docker logs, it says it's trying to convert the files but they never actually get created and there are no errors after multiple attempts. None of these get converted:

photoprism_1  | time="2022-10-25T19:31:58Z" level=info msg="indexing files in 2022/10"
photoprism_1  | time="2022-10-25T19:31:58Z" level=info msg="classify: loading nasnet"
photoprism_1  | time="2022-10-25T19:32:10Z" level=info msg="classify: loading labels from labels.txt"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="index: found no .ppignore file"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="convert: converting IMG_8315.HEIC to IMG_8315.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="convert: converting IMG_8317.HEIC to IMG_8317.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="convert: converting IMG_8319.HEIC to IMG_8319.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="convert: converting IMG_8318.HEIC to IMG_8318.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="convert: converting IMG_8322.HEIC to IMG_8322.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="convert: converting IMG_8323.HEIC to IMG_8323.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:32:11Z" level=info msg="convert: converting IMG_8324.HEIC to IMG_8324.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:32:22Z" level=info msg="indexing completed in 24 s"

I downgraded to the stable release Build [220901-f493607b0] and now the conversion seems to be working:

photoprism_1  | time="2022-10-25T19:43:05Z" level=info msg="indexing files in 2022/10"
photoprism_1  | time="2022-10-25T19:43:05Z" level=info msg="classify: loading nasnet"
photoprism_1  | time="2022-10-25T19:43:17Z" level=info msg="classify: loading labels from labels.txt"
photoprism_1  | time="2022-10-25T19:43:18Z" level=info msg="index: found no .ppignore file"
photoprism_1  | time="2022-10-25T19:43:18Z" level=info msg="convert: converting IMG_8315.HEIC to IMG_8315.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:18Z" level=info msg="convert: converting IMG_8317.HEIC to IMG_8317.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:22Z" level=info msg="convert: IMG_8315.HEIC.jpg created in 4.872088369s (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:23Z" level=info msg="convert: IMG_8317.HEIC.jpg created in 4.951915285s (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:28Z" level=info msg="media: created 11 thumbnails for 2022/10/IMG_8315.HEIC.jpg [5.969757844s]"
photoprism_1  | time="2022-10-25T19:43:29Z" level=info msg="index: added main heif file 2022/10/IMG_8315.HEIC"
photoprism_1  | time="2022-10-25T19:43:29Z" level=info msg="media: created 11 thumbnails for 2022/10/IMG_8317.HEIC.jpg [6.262094483s]"
photoprism_1  | time="2022-10-25T19:43:29Z" level=info msg="index: added main heif file 2022/10/IMG_8317.HEIC"
photoprism_1  | time="2022-10-25T19:43:49Z" level=info msg="index: matched 1 label with IMG_8317.HEIC.jpg [19.335955681s]"
photoprism_1  | time="2022-10-25T19:43:49Z" level=info msg="index: stacked related jpg file 2022/10/IMG_8315.HEIC.jpg"
photoprism_1  | time="2022-10-25T19:43:49Z" level=info msg="convert: converting IMG_8318.HEIC to IMG_8318.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:49Z" level=info msg="index: stacked related jpg file 2022/10/IMG_8317.HEIC.jpg"
photoprism_1  | time="2022-10-25T19:43:49Z" level=info msg="convert: converting IMG_8319.HEIC to IMG_8319.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:54Z" level=info msg="convert: IMG_8319.HEIC.jpg created in 4.592628793s (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:54Z" level=info msg="convert: IMG_8318.HEIC.jpg created in 4.933780148s (heif-convert)"
photoprism_1  | time="2022-10-25T19:43:59Z" level=info msg="media: created 11 thumbnails for 2022/10/IMG_8319.HEIC.jpg [5.588132368s]"
photoprism_1  | time="2022-10-25T19:44:00Z" level=info msg="index: added main heif file 2022/10/IMG_8319.HEIC"
photoprism_1  | time="2022-10-25T19:44:01Z" level=info msg="media: created 11 thumbnails for 2022/10/IMG_8318.HEIC.jpg [6.913901941s]"
photoprism_1  | time="2022-10-25T19:44:01Z" level=info msg="index: added main heif file 2022/10/IMG_8318.HEIC"
photoprism_1  | time="2022-10-25T19:44:03Z" level=info msg="index: stacked related jpg file 2022/10/IMG_8319.HEIC.jpg"
photoprism_1  | time="2022-10-25T19:44:03Z" level=info msg="convert: converting IMG_8322.HEIC to IMG_8322.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:44:04Z" level=info msg="index: matched 2 labels with IMG_8318.HEIC.jpg [2.547982736s]"
photoprism_1  | time="2022-10-25T19:44:05Z" level=info msg="index: stacked related jpg file 2022/10/IMG_8318.HEIC.jpg"
photoprism_1  | time="2022-10-25T19:44:05Z" level=info msg="convert: converting IMG_8323.HEIC to IMG_8323.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:44:08Z" level=info msg="convert: IMG_8322.HEIC.jpg created in 5.341076239s (heif-convert)"
photoprism_1  | time="2022-10-25T19:44:09Z" level=info msg="convert: IMG_8323.HEIC.jpg created in 4.042427045s (heif-convert)"
photoprism_1  | time="2022-10-25T19:44:13Z" level=info msg="media: created 11 thumbnails for 2022/10/IMG_8322.HEIC.jpg [5.261064405s]"
photoprism_1  | time="2022-10-25T19:44:14Z" level=info msg="index: added main heif file 2022/10/IMG_8322.HEIC"
photoprism_1  | time="2022-10-25T19:44:14Z" level=info msg="media: created 11 thumbnails for 2022/10/IMG_8323.HEIC.jpg [5.682590528s]"
photoprism_1  | time="2022-10-25T19:44:14Z" level=info msg="index: added main heif file 2022/10/IMG_8323.HEIC"
photoprism_1  | time="2022-10-25T19:44:16Z" level=info msg="index: matched 2 labels with IMG_8322.HEIC.jpg [2.25973422s]"
photoprism_1  | time="2022-10-25T19:44:17Z" level=info msg="index: stacked related jpg file 2022/10/IMG_8322.HEIC.jpg"
photoprism_1  | time="2022-10-25T19:44:17Z" level=info msg="convert: converting IMG_8324.HEIC to IMG_8324.HEIC.jpg (heif-convert)"
photoprism_1  | time="2022-10-25T19:44:18Z" level=info msg="index: stacked related jpg file 2022/10/IMG_8323.HEIC.jpg"
photoprism_1  | time="2022-10-25T19:44:21Z" level=info msg="convert: IMG_8324.HEIC.jpg created in 4.683319975s (heif-convert)"
photoprism_1  | time="2022-10-25T19:44:25Z" level=info msg="media: created 11 thumbnails for 2022/10/IMG_8324.HEIC.jpg [3.938327288s]"
photoprism_1  | time="2022-10-25T19:44:25Z" level=info msg="index: added main heif file 2022/10/IMG_8324.HEIC"
photoprism_1  | time="2022-10-25T19:44:27Z" level=info msg="index: stacked related jpg file 2022/10/IMG_8324.HEIC.jpg"
photoprism_1  | time="2022-10-25T19:44:38Z" level=info msg="moments: added 'October 2022' (public:true year:2022 month:10)"
photoprism_1  | time="2022-10-25T19:44:41Z" level=info msg="indexing completed in 96 s"

Do you need me to send you one of these photos to figure out what's going on?

@lastzero
Copy link
Member

Yes, samples would be great! See the contact page on our website for details. Thanks ❤️

@inthreedee
Copy link

Sample sent. Upon closer inspection, I see that, in fact, none of my HEIC images were converted on the preview build. All the photos I do see were uploaded with a corresponding jpg because they were either edited or shot in a special mode that created a jpg on the phone.

So, to clarify, that preview build mentioned above is not converting any of my HEIC images. That seems super weird to me based on the discussion above, so I want to make sure it's not a configuration issue on my end. Are there any new settings that need to be changed when switching from the current stable to the preview build for this to work properly?

@graciousgrey
Copy link
Member

@inthreedee I just indexed the sample file on the preview build and PhotoPrism created a sidecar JPG as expected. What settings do you use? Did you disable one of the converters?

@lastzero
Copy link
Member

@inthreedee Is it possible that you use a special Docker image version, e.g. for ARM64, ARMv7 or based on Debian? We have tested our custom heif-convert binary mainly on AMD64 and it is only available for Ubuntu 22.04 LTS so far.

@inthreedee
Copy link

@lastzero Yes, this is on a Raspberry Pi 4 running Ubuntu Server 22.04.1LTS. Let me know what I can do to help.

I haven't disabled any of the converters and sidecar creation seems to work fine when I downgrade to the stable version of PhotoPrism without changing any other settings. If there's a specific setting I need to look at, please let me know

@lastzero
Copy link
Member

Good catch! Might be a library missing or it didn't compile properly. Should have tested the ARM version with an actual HEIC file too... You could turn on trace mode to see the error message in the Docker logs, if any. For security reasons, info logs and higher don't include server internals anymore.

@lastzero
Copy link
Member

Can you verify if you are using the 32-bit ARMv7 version for some reason? It can happen on Raspberry Pi OS as it is a 32-bit operating system.

@lastzero
Copy link
Member

So the binary is there, but it fails due to a missing symbol in a shared library. Trying to find a solution!

lastzero added a commit that referenced this issue Oct 26, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
@inthreedee
Copy link

Can you verify if you are using the 32-bit ARMv7 version for some reason? It can happen on Raspberry Pi OS as it is a 32-bit operating system.

I'm using 64bit Ubuntu Server for the OS with the arm_64bit=1 setting in my config.txt. How do I verify which version of the multi-arch PhotoPrism image was fetched? My docker-compose line doesn't specify one or the other:
image: photoprism/photoprism:preview

lastzero added a commit that referenced this issue Oct 26, 2022
Signed-off-by: Michael Mayer <michael@photoprism.app>
@lastzero
Copy link
Member

A difference is the newer Darktable version in the AMD64 image. Darktable has libheif 1.12 as a dependency, which probably causes our custom heif-convert 1.13 binary to be linked against this old version. It seems that I can fix the problem by setting an explicit LD_LIBRARY_PATH when calling heif-convert with Go. Easy!?

@inthreedee
Copy link

If that's the case, let me know when another preview build is ready and I'll test it on my end for you.

@lastzero
Copy link
Member

@inthreedee Build 221026-7723e6b32 is almost ready and the release notes have been updated. Cross compilation for ARM64 always takes a little longer. Try in a few minutes!

@lastzero
Copy link
Member

Done! Happy testing 😉

@inthreedee
Copy link

Seems to have done the trick, thanks!
I have a few thousand new photos to index now. I'll let that run and report back if anything fails.

@MoweME
Copy link
Author

MoweME commented Oct 31, 2022

Thank you @lastzero for your awesome work.
I also appreciate the help from all the helpers in this issue!

This issue seems to be resolved!

@lastzero lastzero added released Available in the stable release and removed please-test Ready for acceptance test labels Nov 2, 2022
@lastzero lastzero closed this as completed Nov 2, 2022
@inthreedee
Copy link

Sorry for taking so long to report back. Real life got in the way of testing for a bit there.
In any case, just confirming that everything appears to have gone smoothly after importing that latest batch of several thousand new photos. 👍

Thanks again!

@lastzero
Copy link
Member

lastzero commented Nov 4, 2022

Thanks for testing! 🎂

@lastzero lastzero moved this to Released 🌈 in Roadmap 🚀✨ Jun 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working docker Docker Images, Build Scripts, Config & Deployment Examples released Available in the stable release
Projects
Status: Release 🌈
Development

No branches or pull requests

9 participants