Enhancement: support custom (non sRGB) output ICC profiles and colourspaces

[lovell]

This will convert pixel values using a provided ICC profile and attach that profile to the output image metadata.

It should assume the profile is RGB. If e.g. a CMYK profile is required, toColourspace must be used in addition.

Convert to and attach custom RGB output profile (proposed):

sharp(input)
  .withMetadata({ profile: '/path/to/rgb.icc' })
  ...

Convert to and attach existing RGB output profile (proposed):

sharp(rgbInput)
  .toColourspace('rgb')
  .withMetadata()
  ...

Convert to and attach custom CMYK output profile (proposed):

sharp(input)
  .toColourspace('cmyk')
  .withMetadata({ profile: '/path/to/cmyk.icc' })
  ...

Convert to and attach existing (or default if missing) CMYK output profile (proposed):

sharp(cmykInput)
  .toColourspace('cmyk')
  .withMetadata()
  ...

The icc_export, or icc_transform when no input profile, operation can be used for this.

The current existing behaviour, as in the following examples, will remain.

Convert to sRGB without embedded profile (existing behaviour):

sharp(input)
  ...

Convert to and attach existing sRGB output profile (existing behaviour):

sharp(input)
  .withMetadata()
  ...

See #1324 for custom input profile support.

See #218 for previous discussion on this and related features.

See #734 for a related enhancement that this might address.

[caesar]

Is this the correct place to 👍 for supporting the use-case of keeping the original (in my case CMYK) colourspace and not performing any conversions at all?

[lovell]

For CMYK input and output I think the input image would be internally converted to a wider gamut, either sRGB or scRGB (see #1317), any processing applied, then converted back to CMYK using the original profile (or a default CMYK profile if none was embedded).

I'm unsure how well the resizing/resampling kernels will work directly with subtractive CMYK, plus other features like overlay composition might also become more complex and/or not take optimised RGB(A) code paths within libvips.

[caesar]

Wouldn't that result in losses / changes due to converting from CMYK to RGB and back again?

[lovell]

Yes, there is a potential for small rounding/clipping losses.

The CMYK model uses a narrow gamut so I would guess that any loss to/from colourspace conversion to a wider gamut is less bad than repeated clipping during operations such as resize/composite.

Internal use of the 16-bit scRGB space would include pretty much all possible 8-bit CMYK values and would ensure the internals of sharp are simpler to maintain.

Should the output CMYK profile be different to the input CMYK profile, e.g. switching from SWOP to ISO/FOGRA, then I'd expect internal conversion to/from a wider gamut could improve precision.

[e-tip]

Hi @lovell ,
there's an ETA on this ? we'd like to move from srgb images to Adobe RGB

[roborourke]

I realise this isn't the proposed approach in terms of API but I have a working (I think) prototype on this branch: https://github.com/roborourke/sharp/tree/icc-transform

It adds a withIcc() method to the sharp object. The ICC profile is applied using icc_transform() right before output & metadata handling.

Here are the results with one of the test images:

original	hilutite.icm (from lutify.me applied)

[chaddjohnson]

Hi! I tried the following, but the resulting file is still in sRGB (the input image is in CMYK):

return sharp('./input.tif')
    .resize(18000, 26046, {fit: 'fill'})
    .toColourspace('CMYK')
    .withMetadata({profile: './USWebCoatedSWOP.icc'})
    .toFile('./output.tif')

Seems this functionality is still only a proposal.

We've been using ImageMagick, and it is a memory hog with large images. I've tried using it and GraphicsMagick with Lambda to resize large images, but we constantly run into memory issues, and the Lambda functions fail.

Sharp is powerful and seems to use less memory, and it would be absolutely great if this library allowed users to retain the original color profile without performing any conversions (i.e., do not convert to sRGB and then back to CMYK). With this functionality, this library would be super useful in the Print on Demand market space since everything is printed in CMYK. Without this, Sharp unfortunately is entirely useless for print images.

[roborourke]

I’ll pick up my branch again and work on adding it in the way proposed here rather than `withIcc()`

…

On Thu, 3 Jan 2019 at 18:23, Chad Johnson ***@***.***> wrote: Hi! I tried the following, but the resulting file is still in sRGB (the input image is in CMYK): return sharp('./input.tif') .resize(18000, 26046, {fit: 'fill'}) .toColourspace('CMYK') .withMetadata({profile: './USWebCoatedSWOP.icc'}) .toFile('./output.tif') Seems this functionality is still only a proposal. We've been using ImageMagick, and it is a memory hog with large images. I've tried using it and GraphicsMagick with Lambda to resize large images, but we constantly run into memory issues, and the Lambda functions fail. Sharp is powerful and seems to use less memory, and it would be absolutely great if this library allowed users to retain the original color profile without performing any conversions (i.e., do not convert to sRGB and then back to CMYK). If we had this, this library would be super useful in the print on demand market space since everything is printed in CMYK. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1323 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABbeQaKInCHnj1bXkn7BqQglOC03m-Rks5u_kq3gaJpZM4VvX4b> .

[randyridge]

Any movement on this? I'd simply like to replace the sRGB.icc with some Compact ICC Profiles in certain scenarios...

[Kodiakkb]

+1 for this, I just need to add the sRGB profile to an image with no profile at all for printing purposes...

[chaddjohnson]

I am still a bit confused why support for CMYK is lacking with Sharp when it is available with vips. For example, I can convert a CMYK TIFF image to CMYK JPEG with no problems using vips:

hyperion:sharp chad$ identify sample1.tif 
sample1.tif TIFF 18000x26046 18000x26046+0+0 8-bit CMYK 84.2091MiB 0.000u 0:00.000
hyperion:sharp chad$ vips jpegsave -Q 100 --optimize-coding sample1.tif sample1.jpg
hyperion:sharp chad$ identify sample1.jpg 
sample1.jpg JPEG 18000x26046 18000x26046+0+0 8-bit CMYK 40.4277MiB 0.000u 0:00.001

However, with

sharp('sample1.tif').limitInputPixels(false).jpeg().pipe(writeStream);

the resulting file is sRGB. And

sharp('sample1.tif').limitInputPixels(false).toColourspace('cmyk').jpeg().pipe(writeStream);

results in

Error: vips_colourspace: no known route from 'srgb' to 'cmyk'

Any ideas why there is a hangup, and is there a roadmap item defined for enabling working with CMYK using Sharp? It would be nice if no conversion to RGB occurs when the source file is CMYK.

[kevinswarner]

It seems there has been much discussion on this issue for over 2 years. Is this still being considered? We would desperately like to move away from IM for processing of our large files. Sharp/VIPS is much more performant. But, we must be able to either maintain images in CMYK or get them back to CMYK with a provided profile, hopefully without too much loss or difference in the final resulting image. Are there any specific reasons for this not moving forward? Just curious as we don't want to spin our wheels too much going down the Sharp path if this is not going to be added. Just looking for some guidance on the probability or timeline of having this capability. Thanks!

[lovell]

Every issue tagged as an enhancement, such as this, represents a future possible enhancement; there is no "ETA", "roadmap" or "timeline".

On behalf of all sharp users I am always very grateful to anyone that helps implement such enhancements, either by using their own time to work on and submit a pull request or by offering to pay for my time to do likewise.

[kevinswarner]

Of course. I completely understand. I may have misunderstood in that I thought it was already being worked on by others. Thank you for all the work and this library. We will take a look to see if there is a way we can help if we cannot find an alternate solution.

[rawpixel-vincent]

I read the proposals, and I'm not quiet sure my use case is listed,
I need to resize a tiff while keeping the original color profile (which is Adobe RGB 1998)

Currently, using the following give me an sRGB color profile in the output instead of maintaining the original color profile

sharp(input)
  .withMetadata()
  ...

[roborourke]

I picked up my work on this again and I've created a pull request #2271 - it's work in progress and I need a little guidance to properly implement everything mentioned in the original comment.

For now it accepts withMetadata({ profile: 'custom.icc' }) and applies that profile just before output so toColourspace() should still be applied before it.

[roborourke]

I read the proposals, and I'm not quiet sure my use case is listed,
I need to resize a tiff while keeping the original color profile (which is Adobe RGB 1998)

Currently, using the following give me an sRGB color profile in the output instead of maintaining the original color profile

sharp(input)
  .withMetadata()
  ...

The implicit conversion to sRGB is intentional, I believe the library is specifically geared towards web use and so defaults to providing the smallest possible file size by normalising to a device independent web safe colourspace and stripping the ICC profile. The sRGB conversion will always occur but I think this change will let you convert to and add the original colour profile back in.

[lovell]

#2271 has landed (thanks @roborourke) and will provide the following API:

// Available from v0.26.0
sharp(input).withMetadata({ icc: '/path/to/profile.icc' })...

This should meet the needs of two parts of this enhancement, namely "Convert to and attach custom RGB output profile" and "Convert to and attach custom CMYK output profile".

[hdwong]

I found the same problem. I have a jpeg image with CMYK mode and it does not have ICC profile embedded in the metadata,

sharp('path-to-original.jpg').toFile('path-to-result.jpg');

I tried any of the solutions above without getting the correct result.
(The background color of the original image is pure black, but the background is brighter than the original in the result image)

original	Load and save by Sharp

[roborourke]

@hdwong can you make sure you’re using version 0.26.0 and try modifying your code to keep the CMYK profile:

sharp('path-to-original.jpg')
  .withMetadata({ icc: 'cmyk' })
  .toFile('path-to-result.jpg');

[hdwong]

@roborourke I just want to convert the image from CMYK to sRGB mode. I had upgrade Sharp to version 0.26.0 but the problem still exists.

Now I have used Little CMS command jpgicc to solve this problem, but I still hope to solve this problem with Sharp.

const context = sharp('path-to-original.jpg');
const { format, space } = await context.metadata();
if (format === 'jpeg' && space === 'cmyk') {
  spawnSync('jpgicc', [
    '-i',
    'path-to-cmyk.icc',
    '-o',
    'path-to-srgb.icc',
    'path-to-original.jpg',
    'path-to-result.jpg',
  ]);
}

[lovell]

@hdwong Please can you open a new issue for this, with all the original images, profiles and complete code you're using (for CMYK input you may need to use a zip file or similar to avoid GitHub converting the images).

[hdwong]

@lovell I have created an issue #2365 for this, thanks you for your reply.

[greatestview]

Hey @lovell, first of all: Thank you for your great work!

I have a question regarding the current implementation of color profile conversion: As far as I’ve read #218 and this Issue (and especially your comment #1323 (comment)), it is already possible to convert an image to a specific color profile like so:

sharp(input)
  .withMetadata({ icc: '/path/to/profile.icc' })
  …

I’m currently working on a project, where we need to keep embedded RGB color profiles (especially P3 for iOS devices) and fall back to sRGB, if no profile is embedded. This feature is not yet implemented, right? As stated in the issue description, the proposed code would look like this:

sharp(rgbInput)
  .toColourspace('rgb')
  .withMetadata()
  …

Until we get there, I’ve had an idea of a workround:

const image = sharp(rgbInput)
const { icc: iccBuffer } = await image.metadata()
const iccFile = somehowConvertIccBufferToTempFile(iccBuffer)
image
  .withMetadata({ icc: iccFile })
  …

I’m not very familiar with the sharp internals and ICC file structure, but would that work?

[lovell]

@greatestview Yes, your approach looks like it should work.

[tars-mj]

Hello
I am creating a printing tool and I have a color problem. When I convert a file several times through the sharp library, it recalculates the colors every time even though there is the same color space and the same profile: Fogra39. Anyway, no matter what the profile is there, because even if I give any other profile, the effect is similar. Here is the original file, then saved the third time and finally the fifth time:

My question is, is there any way to use sharp in such a way that it does not interfere with the colors when opening and saving? it's like opening and saving one file in Photoshop every time and there are different colors every time.

Sharp perfectly fits my project needs, the only problem is the colors :)

[wereHamster]

I want to retain the original ICC profile in the output image, and use sharp only to resize and convert to a particular output format (jpeg, avif, webp etc). That feature was requested in #3339.

The trick described in this comment (extracting the ICC profile from the input image, writing it out into a file, and the passing it to withMetadata()) does not work when the input image is using «Apple Wide Color Sharing Profile». That profile seems to be attached to photos made by an iPhone.

The issue is that the profile can only be used as input but not output, see details over here: mm2/Little-CMS#188. But sharp is not smart enough to skip the ICC transform altogether if the input and output profiles are the same. It will still try to run the transform and fail.

What would really be needed is an option, perhaps to withMetadata(), to say: do not run any ICC transform (neither to sRGB or any other target profile) AND attach the original ICC profile to the output.

sharp("input.jpg")
  .resize({ width: 100 })
  .withMetadata({ doNotTransformColorSpaceAndCopyICCProfile: true })
  .toFile("output.jpg");

[uhthomas]

I'm feeling a little confused from following these few linked issues. I'm looking to create thumbnails from photos with AdobeRGB and have found the resulting files are really dull. Is there any way to avoid this?

[TobiasNoell]

I also agree that it is kind of confusing how it is handled currently.
It would be highly desirable to be able to embed an ICC color profile without transforming the data.

[greentore]

It's seriously annoying that there's no straightforward way to keep the input colourspace intact. I think a constructor option like keepColourspace: true would be the best.

[lovell]

@greentore I'm sorry to hear that you are seriously annoyed. If these symptoms persist, may I suggest you create a new question and provide more information about your scenario and the problem you're trying to solve.

[lovell]

For those who weren't already aware, there is a new keepIccProfile feature in v0.33.0 that allows the input ICC profile to be retained in the output image, which probably covers a couple of the scenarios mentioned in comments on this issue.

https://sharp.pixelplumbing.com/api-output#keepiccprofile

[donmccurdy]

I'm interested in being able to assign (not convert) common color spaces when encoding an image, where I know the color profile embedded in the image is unreliable and should be ignored. I'd imagine an API like:

await sharp(input)
 .assignColourspace('srgb')
 .toFile('output.png')

Would that fit with the goals of this thread, and would a PR be welcome if so?

The two main profiles I have in mind are "sRGB" (sRGB transfer, Rec. 709 primaries, D65 white point) and "not a color". I suppose the latter would need to be encoded as "rgb"... I'm not sure how to represent that as an ICC profile.

[lovell]

where I know the color profile embedded in the image is unreliable and should be ignored

@donmccurdy Did you see the ignoreIcc constructor option?

await sharp(input, { ignoreIcc: true })...

[donmccurdy]

@lovell I had not, thank you! I'm trying to fix an incorrect embedded color profile in my output. It sounds like combining ignoreIcc with toColorspace might let me assign a color space without applying any conversion ...

await sharp(input, {ignoreIcc: true})
 .toColourspace('rgb')
 .toFile('output.png')

... but (as of v0.32.6) I'm seeing vips_colourspace: no known route from 'srgb' to 'rgb'.

[lovell]

@donmccurdy The "rgb" colourspace is device-dependent and requires a profile. To force sRGB but not attach a profile, perhaps try something like:

- .toColourspace('rgb')
+ .withIccProfile('srgb', { attach: false })

[donmccurdy]

Hm, thank you — I must be misunderstanding something. I'm probably off-topic here, so I've opened a separate question:

• Assigning metadata to color and non-color images #4011