Colors: Add libvips support to preserve ICC profiles in thumbnails

[bobobo1618]

As a user who has pictures with color profiles other than sRGB, I want PhotoPrism to preserve them in the generated thumbnails so that the colors are properly reproduced on devices/displays that support ICC profiles.

Since thumbnail generation with libvips seems mature enough and we haven't noticed any particular issues or downsides, we've decided to make it the default in our upcoming release without waiting any further. We have updated our preview build so that you can test the latest changes and improvements:

• https://docs.photoprism.app/release-notes/#development-preview
• https://docs.photoprism.app/getting-started/updates/#development-preview

The following config options will be available with our upcoming release (or when using the preview build):

Environment	CLI Flag	Default	Description
PHOTOPRISM_THUMB_LIBRARY	--thumb-library	auto	image processing LIBRARY to be used for generating thumbnails (auto, imaging, vips)
PHOTOPRISM_THUMB_COLOR	--thumb-color	auto	standard color PROFILE for thumbnails (auto, preserve, srgb, none)
PHOTOPRISM_THUMB_FILTER	--thumb-filter	auto	downscaling filter NAME (imaging best to worst: blackman, lanczos, cubic, linear, nearest)
PHOTOPRISM_THUMB_SIZE	--thumb-size	1920	maximum size of pre-generated thumbnails in PIXELS (720-7680)
PHOTOPRISM_THUMB_SIZE_UNCACHED	--thumb-size-uncached	7680	maximum size of thumbnails generated on demand in PIXELS (720-7680)
PHOTOPRISM_THUMB_UNCACHED	--thumb-uncached		generate missing thumbnails on demand (high memory and cpu usage)
PHOTOPRISM_JPEG_QUALITY	--jpeg-quality	83	higher values increase the image QUALITY and file size (25-100)
PHOTOPRISM_JPEG_SIZE	--jpeg-size	7680	maximum size of generated JPEG images in PIXELS (720-30000)
PHOTOPRISM_PNG_SIZE	--png-size	7680	maximum size of generated PNG images in PIXELS (720-30000)

If you build from source or use one of our binary installation packages, please note that the system on which you build and/or run PhotoPrism must have libvips >= 8.10 installed.

---

Original Issue Report

Steps to reproduce:

• Obtain image with an ICC profile applied (e.g. AdobeRGB)
• Import into Photoprism
• Navigate to photo in Photoprism
• Visually compare to original

Expected result: Lower resolution, identical colours.

Actual result: Lower resolution, very noticeable reduction in saturation.

Downloading the thumbnail image and comparing to the original with exiftool shows that the thumbbnail lacks the ICC profile of the original. Copying the ICC profile from the original to the new resolves the problem:

exiftool -icc_profile -b -w icc file.jpg
exiftool "-icc_profile<=file.icc" fit_2048.jpg

[bobobo1618]

As for a solution, it looks like somebody had a similar problem in the past and ended up using libvips via golang bindings. The most mature of these bindings at the moment seems to be govips.

[lastzero]

Great, one more dependency 😂 can take a look at this when we're done with facial recognition and batch edit...

[bobobo1618]

Hmm, another option might be ImageFlow Server. Point it at a bucket of images and you can have it process them with URL query parameters.

That would limit PhotoPrism's role to generating the original previews. It could also potentially write the previews to some cloud storage which ImageFlow could read from, going some way towards facilitating cloud storage, something I've seen asked about a few times.

[lastzero]

related issue with videos: see #1765

[lastzero]

This change converts colors to standard sRGB if an Apple "Display P3" color profile is detected:

• 5be456a

Not sure if color profile conversion also works when originals are HEIF images, needs to be tested.
Videos and file formats other than JPEG haven't been touched yet.

Adding support for additional color profiles should be easy (we can leave that to external contributors).

A small challenge may be profile name comparison: For example "Adobe RGB (1998)" vs just "Adobe RGB". There may be differences or there could be different names for the same profile... research needed.

I'll start a new Development Preview build for testing:

• https://docs.photoprism.org/release-notes/#development-preview

[bobobo1618]

I think it'd be preferable to preserve the colour profile. If you change it to sRGB, information is lost. Displays, devices and software these days support P3 or AdobeRGB more often than not so if you convert, you end up with a situation where a user sees one set of colours on their device, then a different set of colours in Photoprism.

[lastzero]

Why did I even think you would be happy about the progress? 🤣

As I've said many times: We're focusing on multi-user support right now, in addition to handling tons of support emails and other questions - so this unexpected improvement is definitely something to be excited about! ⚠️

The sRGB color space is widely used for files, displays, printers, and other devices. Almost all displays can reproduce sRGB or something close to it. If you have time, please provide us with an overview which apps, devices & printers have native support for ICC Profiles and Apple Display P3 in particular (or more importantly, which apps and devices do NOT) 👍

Keep in mind: When there is just one device that doesn't support Display P3, you end up maintaining two sets of thumbnails. It's the same as with AVC: Some browsers support more recent codecs and they may have slightly better details, but storing (and transcoding to) multiple formats is a huge overhead and adds complexity to no end.

According to my information, colors should be converted anyway before downscaling since the algorithms are not designed for P3 colors, which may cause artifacts. Update: Found the links again to this excellent article and this.Turns out, Display P3 and sRGB both use a gamma of 2.2 so for these two profiles there might be no major difference, only the professional DCI-P3 uses 2.6. Still, simply adding metadata to thumbs won't get you the best possible quality and may cause issues with older computers (yes, old people and computers still exist). There are tradeoffs everywhere. Let's spend more time with research!

[bobobo1618]

Don't get me wrong, it's definitely an improvement! I just think that for me, it's a move in the wrong direction, since the problem (seeing different colours in Photoprism to the ones I see on my phone) is still there, it's just different now.

If you have time, please provide us with an overview which apps, devices & printers have native support for ICC Profiles

ICC profiles are software dependent, not device or printer dependent. It's up to the software displaying the image to handle it in the way the "display" (including printers here) expects.

They're supported on all major browsers, and on Android and iOS (where they're part of the OS, so should be supported in ~all apps).

One exception is ICC v4, which is needed for the normal Display P3 profile. This has fairly wide support but isn't supported on Firefox on Windows (works on Mac and Linux though).

There is this project which provides an ICC v2 substitute for Display P3, however.

So my suggestion would be:

• If the image has an ICC v2 profile, leave it. Support for ICC v2 is near-universal.
• If the image has an ICC v4 profile and you're worried about Firefox support on Windows, convert to sRGB but maybe flag it somewhere like in image metadata details so the user can discover the problem organically.
• One day, maybe think about converting to ICC v2 equivalents of ICC v4 profiles, or storing a v2 or sRGB fallback thumbnail for images with v4 profiles.

Keep in mind: When there is just one device that doesn't support Display P3, you end up maintaining two sets of thumbnails. It's the same as with AVC: Some browsers support more recent codecs and they may have slightly better details, but storing (and transcoding to) multiple formats is a huge overhead and adds complexity to no end.

Does that make sense in the scope of Photoprism though? It's currently 100% browser-based. Does it matter if you can put a Photoprism thumbnail on an SD card and stick it in a digital photo frame or printer?

According to my research, colors should be converted anyway before downscaling since the algorithms are not designed for P3 colors, which causes artifacts.

I mostly switched to Piwigo after this issue and haven't noticed any problems. It might be a problem with super tiny images or if you're really pixel peeping but practically I don't notice anything wrong.

[lastzero]

ICC profiles are software dependent, not device or printer dependent. It's up to the software displaying the image to handle it in the way the "display" (including printers here) expects.

Yes, so it's out of our control. If it doesn't work, users blame us first just like you do - not their software.

One exception is ICC v4, which is needed for the normal Display P3 profile.

Display P3 is the single biggest issue as Apple seems to use it now by default, so all Apple users are affected. That's why I've spent extra time providing a "good enough" solution while we give more sponsors time to join the party. Quid pro quo.

There is this project which provides an ICC v2 substitute for Display P3, however.

We have too much work already. No hacks please.

Does that make sense in the scope of Photoprism though? It's currently 100% browser-based. Does it matter if you can put a Photoprism thumbnail on an SD card and stick it in a digital photo frame or printer?

Yes, it's a huge business and people do this. Ever heard of photo books? 😂

I mostly switched to Piwigo after this issue and haven't noticed any problems.

So you already have a good solution. That takes some pressure off us! We will come back to ICC Profiles when other high priority items on our roadmap are done :)

[bobobo1618]

Yes, so it's out of our control. If it doesn't work, users blame us first just like you do - not their software.

There are multiple ways it can "not work":

• The client software might not support the ICC profile. In this case the image might just look a bit off (99% of cases) or it might look totally crazy (for very exotic ICC profiles).
• PhotoPrism removes or converts the image colours. In this case the image will, 100% of the time, look wrong.

Clearly, you have no control over the rare and minor first case but you have total control over the second.

Yes, it's a huge business and people do this. Ever heard of photo books? 😂

Sure but why on earth would you make a photobook from a thumbnail?

[lastzero]

Take it easy! We will get to it, just not today unless you get us more developers.

To be honest, I can't tell ANY difference on my mid-range 25" Dell screen and my Huawei P30 phone. Could be because I'm too old and my hardware is still too cheap, granted.

I use high quality resampled images for almost everything so that the size fits the format. For my needs, that has the highest impact. Most services and social media sites do a terrible job with downsampling. Speaking of it, do all social media apps and sites support v4 profiles? This kind of information would really help, not your ranting.

I generally don't want my metadata spread all over the Internet. A side effect is that ICC Profiles are currently also missing. When we've developed this, Display P4 wasn't used by default - at least not that I can remember.

[lastzero]

In addition, we need somebody to do research on how to get the ICC Profiles in the thumbs. It seems the library we use for READING doesn't cover this:

• https://github.com/mandykoh/prism

Using a Perl script (as suggested in the issue description) or an external C-library isn't a great solution. It may please your luxury eyes, but it's more difficult to maintain, comes with a performance overhead, and also gives me pimples.

[lastzero]

BTW: Is it possible to embed custom profiles or additional parameters? Seems like a great way to track images without raising attention 🕵️

[lastzero]

Now that we have this informative thread, I'll create a new issue for the current improvements and save this for later 👍

[bobobo1618]

Take it easy! We will get to it, just not today unless you get us more developers.
This kind of information would really help, not your ranting.

I think you might be reading frustration/aggression/impatience where there is none. I'm not upset with you or anything (I'd have absolutely no right to be if that were the case), I'm just trying to elaborate where it looks like you might not fully understand something.

As for the "ranting" point, I've been trying to act in good faith, I've done all the research you asked me to do and I think my suggestions have been reasonable given the information available to me at the time so it's really disappointing that you seem to see me as "ranting".

To be honest, I can't tell ANY difference on my mid-range 25" Dell screen and my Huawei P30 phone. Could be because I'm too old and my hardware is still too cheap, granted.

Have a look at the webkit wide gamut test on a device with a wide-gamut display (e.g. a recent iPhone). The "Iceland" and "Italy" examples have some pretty prominent differences in P3 vs. sRGB.

Speaking of it, do all social media apps and sites support v4 profiles?

"All" is a very rigid word but Instagram for example has supported Display P3 since 2017, same as 500px. Flickr supported it since at least 2007, Google Photos seems to have added it in 2019 (though it's unclear whether that counts backups).

Not sure about Facebook or whatever else the kids are using these days but I expect on sites that don't really focus on photos, the treatment is whatever's easiest at the time.

I generally don't want my metadata spread all over the Internet.

I agree and I'd consider most metadata to be a bug, even timestamps if they're sufficiently precise.

Using a Perl script (as suggested in the issue description) or an external C-library isn't a great solution. It may please your luxury eyes, but it's more difficult to maintain, comes with a performance overhead, and also gives me pimples.

To be clear, the exiftool example wasn't intended as a proposed solution, just a way to verify that the problem was a ICC profile and not something else.

As for libvips, I do see the problem from the perspective of preferring a self-contained Go binary. Aside from being a dependency itself, it has a number of other dependencies on libraries used for reading image formats and the like.

But what are the constraints you want to impose on dependencies? There's going to be some dependency no matter what, even if it's just another pure Go package. Have you documented your preferences anywhere (e.g. licensing)?

Aside from maybe FreeBSD, it seems the only supported deployment method for PhotoPrism is Docker and looking at the image, there are already a ton of C dependencies. So why not add another? (To be clear that I don't intend to be belligerent here, I mean this in the sense of "what is the very good reason I'm sure you have", not "you're just being silly and I'm poking you to make you admit it")

BTW: Is it possible to embed custom profiles or additional parameters? Seems like a great way to track images without raising attention 🕵️

Yes it is and yes, that's a potential risk. I'm not sure that it's worth defending against though. If image fingerprinting is the goal, better to put it in the image data itself à la Digimarc.

[lastzero]

Based on tests performed with the "photoprism thumbs -f" command, libvips requires twice as much time than the native Go implementation we currently use. Changing the cache and concurrency settings didn't have much of an effect, so I guess we need to optimize it in the same way as the native rendering - by generating smaller sizes from existing thumbs rather than the original image files.

[lastzero]

With these changes, thumbnail rendering performance is 42% better with libvips than with our native Go implementation. So I think we can release this, unless someone finds another issue that should be resolved?

[lastzero]

Since thumbnail generation with libvips seems mature enough and we haven't noticed any particular issues or downsides, we've decided to make it the default in our upcoming release without waiting any further. We have updated our preview build so that you can test the latest changes and improvements.

If you build from source or use one of our binary installation packages, please note that the system on which you build and/or run PhotoPrism must have libvips >= 8.10 installed.

[johnappletree]

This is really cool! Thank you for this feature. Are video thumbnails also generated using libvips? Live pictures thumbnails look fine now but video thumbnails still look whitewashed.

[lastzero]

@johnappletree we use FFmpeg to extract still images from video files:

photoprism/internal/photoprism/convert_image_jpeg.go

Lines 29 to 33 in 243fa3e

	// Extract a still image to be used as preview.
	if f.IsAnimated() && !f.IsWebP() && c.conf.FFmpegEnabled() {
	// Use "ffmpeg" to extract a JPEG still image from the video.
	result = append(result, exec.Command(c.conf.FFmpegBin(), "-y", "-ss", ffmpeg.PreviewTimeOffset(f.Duration()), "-i", f.FileName(), "-vframes", "1", jpegName))
	}

Unfortunately, we didn't have time to test if FFmpeg preserves color profiles. It is possible that this requires an additional command flag or a newer version. Our stable release is based on Ubuntu 24.04 LTS, so we are currently using FFmpeg 6.1.1. However, we would like to upgrade to FFmpeg 7 as soon as possible.