How can I load an (sRGB-)image in HSV-color space?

[Gewerd-Strauss]

Hello,

I am quite confused about how to control which color-space imager::load.image() loads images, or rather about how to properly convert them to a desired color-space.

Assume the attached jpg-image is loaded via imager:

path <- r"(example.jpg)"
im <- imager::load.image(path)
im_srgb_to_rgb <- imager::sRGBtoRGB(im)
im_rgb_to_hsv <- imager::RGBtoHSV(im_srgb_to_rgb)
imager::display(im)
imager::display(im_srgb_to_rgb)
imager::display(im_rgb_to_hsv)

Original sRGB-Image on disc, saved as 'example.jpg':

Image as loaded and displayed via imager::display(im) (subset):

Image after srgb2rgb:

Image after rgb2hsv:

This obviously does not seem to be right - at least to my eyes; the first transformation from SRGB to RGB works properly.
However, then converting to HSV results in the mess we see above. This also occures if we convert the loaded im directly to HSV. Note that imager::spectrum(im) returns 3, so the originally loaded images does have 3 channels. I would assume that therefore I should be able to convert SRGB > RGB > HSL.

What exactly am I missing?
Any help is welcome.

Thank you.
Sincerely,
~Gw

---

As background, I must detect yellow & green pixels within comparable images; and from my understanding I'd much rather do so in HSV-color space than in RGB, because color-detection in RGB is rather annoying.
So the actual question really is how I can load an image as HSV as resource-efficient as possible, without apparently nuking the image?

[asgr]

This isn't really code I am familiar with (whilst we now maintain imager, we obviously didn't write much of the code base, and do not use all of its functionality). That said- from what I can see the display function expects images to be in a RGB colour space, and sRGB is just close enough that visually it still looks reasonable. So, I don't think there is actually a bug here, it is just being used incorrectly when trying ti directly display HSV. That's my quick assessment- I might be wrong.

The display function itself is pretty low level (Cimg itself) so any bugs there should be communicated to the Cimg team (since we just use their header for imager).

[asgr]

Although now I realise I cannot even display the images like this on my Mac version. It complains I don't have the X11 package installed (which I don't, but that isn't needed anymore and isn't available for R now). How did you get round that?

[Gewerd-Strauss]

It complains I don't have the X11 package installed (which I don't, but that isn't needed anymore and isn't available for R now). How did you get round that?

I don't have an x11-package installed either. Are there any additional potential packages that might be at play here? However, bear in mind that at least the github-source of the package lists X11 as a system-dep:

imager/DESCRIPTION

Line 25 in 6586883

SystemRequirements: fftw3, libtiff, X11, C++11

Which bears the question of how I am satisfying this dep; because I sure don't recall setting it up - nor can I find it anywhere ¯\_(ツ)_/¯

I have created a small R-project that reproduces the issue on my end. Additionally, it contains the image in question - just in case that uploading to and downloading from a github-issue alters it (it shouldn't, but I've seen weirder shit). All packages installed during installation of imager are recorded in the lockfile and can be restored via renv:
https://github.com/Gewerd-Strauss/reprex_clean_imager

[Gewerd-Strauss]

So, I don't think there is actually a bug here, it is just being used incorrectly when trying ti directly display HSV. That's my quick assessment- I might be wrong.

In that case, a couple of things:

1. Can I trust the numerical array resulting from

im <- imager::load.image(path)
im_srgb_to_rgb <- imager::sRGBtoRGB(im)
im_rgb_to_hsv <- imager::RGBtoHSV(im_srgb_to_rgb)

(disregarding the small info-loss between srgb and rgb?). Does im_rgb_to_hsv actually contain proper hsv-formatted data?

2. If I can't trust the hsv-data, I assume my only options are to either

   1. implement a very slow translational alg in R myself to arrive at valid hsv-data?
   2. process in RGB and just hope the issue doesn't compound
      ?

3. Are you aware of any other packages that can convert srgb/rgb<>hsv?

4. Do you know which format imager::load.image loads in? It seems to be either sRGB or RGB, but I am not sure.

[asgr]

I don't think the issue is in the load, but rather the display- that function only meaningfully displays RBG data. Again, this is just from having a cursory look at the lower level calls being made- the display itself is really handled by the Cimg library not imager.

[rtobar]

@Gewerd-Strauss what @asgr says seems to be right to me too. You are giving HSV values to display, which is interpreting them as RGB, so you don't get what you expect.

I think it's safe to trust the HSV data. Given how the underlying CImg library is used so widely in many products, and this being is one of the core functions that the library would perform, you'd expect it to perform it correctly. If you want to double-check you can compare a couple of individual HSV pixel values in im_rgb_to_hsv against the RGB values in im_srgb_to_rgb, and check with an external converter tool (first google hit shows https://www.rapidtables.com/convert/color/rgb-to-hsv.html) that the values correspond.

Alternatively you can alter the HSV values in a meaningful way, the display the HSV->RGB result and check that it makes sense. For example, you do im_rgb_to_hsv[1:3000,,1,3] = 0 and check that the first half of your image is blacked out. Or something along those lines.

[Gewerd-Strauss]

Alternatively you can alter the HSV values in a meaningful way, the display the HSV->RGB result and check that it makes sense. For example, you do im_rgb_to_hsv[1:3000,,1,3] = 0 and check that the first half of your image is blacked out. Or something along those lines.

I did so today, and am happy to confirm that the experiment seems to confirm that conversion is indeed valid. Or to be more precise - it is visually blacked out.

Unfortunately, there seem to still be some issues:

Let's inspect the original image im. Let's take an arbitrary pixel (2674,3872), color space: sRGB:

I use GIMP to retrieve this info. GIMP uses its built-in sRGB space.

Now, let's see what imager loads; im <- imager::load.image(path) returns object im, which should contain pixels in sRGB-colorspace.
For r im[2674,3872,1,X], we get in sRGB:

Flipping the coordinates gives similar values:

If we inspect the RGB-colors after sRGB>RGB-conversion:

Flipped:

This means that GIMP either

1. internally loads the image in sRGB, but reports pixels in RGB instead of sRGB. In this case, the data reported by GIMP should line up with im_srgb_to_rgb[].
2. internally loads the image in sRGB and also reports in sRGB (and the description in the GIMP-GUI is a misnomer). In this case, the data should line up with im.

Neither of those would line up even remotely on scales of either 0-1 or 0-255 with where they should be.

Taking an arbitrary yellow pixel gives the same issue:

We get a similar issue; in that values don't seem to line up.

I did so for a couple more pixels, but don't see the need to paste another 30 cropped screenshots in here.

The hex-codes identified by GIMP are also returned by various other means when inspecting these pixels. Either GIMP lies, along with MS Paint, Autohotkey 1 & 2, and virtually every other utility I could find that can report pixel info; or the color-conversion of the R-package is somewhat fishy.

To me, the values reported make little sense, and don't line up with colors and hex codes reported by any other program or converter.

This leaves me utterly confused, because now basically nothing seems to make any sense. I kinda hope that I am just completely unaware of something here.

There are further oddities that don't make too much sense imo, a curious example:

path <- r"(D:\Dokumente neu\Repositories\clean_imager\example.jpg)"
im <- imager::load.image(path)                                      ## load image
im_srgb_to_rgb <- imager::sRGBtoRGB(im)                             ## convert srgb to rgb
im_rgb_to_hsv <- imager::RGBtoHSV(im_srgb_to_rgb)                   ## convert to hsv
im_rgb_to_hsv_altered3 <- im_rgb_to_hsv                              ## create copy
im_rgb_to_hsv_altered3[1:3000,1:4000,1,3] = 1                        ## alter val
im_hsv_to_rgb_altered3 <- imager::HSVtoRGB(im_rgb_to_hsv_altered3)    ## convert back to RGB
imager::display(im_hsv_to_rgb_altered3)

The image is 6000x4000 pixels large. Thus, im_rgb_to_hsv_altered3[1:3000,1:4000,1,3] = 1 should only modify half the image - the left half, to be specific.

However, this is the image we get:

Given the ranges used 1:3000 & 1:4000, how are all pixels of the image altered? For reference, here is the HSV-image before modifying pixels, via imager::display(imager::HSVtoRGB(im_rgb_to_hsv)):

---

So yea. I have no clue what the hell is going on. ¯\_(ツ)_/¯

[asgr]

Without getting into the colour space conversion issue (you might need to raise questions there with Cimg directly), the jpeg I inspect is identical between imager::load and jpeg::readJPEG. Note readJPEG uses a different structure where the x/y dimensions are swapped (neither is wrong, since matrices we display matrices in a transposed sense anyway).

test_imager = imager::load.image('~/Downloads/309751145-e22f4886-1473-4925-977e-86dbbe5ab1db.jpg')
test_imager[2996,3261,1,]
[1] 0.3019608 0.2980392 0.2823529

test_base = readJPEG('~/Downloads/309751145-e22f4886-1473-4925-977e-86dbbe5ab1db.jpg')
test_base[3261,2996,]
[1] 0.3019608 0.2980392 0.2823529

readJPEG does warn that other programs flip the ink scales. There are also many ways other programs might remap RGB (for reasons of dynamic range). Both imager::load and jpeg::readJPEG access the raw data as encoded in the file.

[rtobar]

@Gewerd-Strauss I think your coordinates are also wrong? The arbitrary pixel with RGB = (46, 125, 96) is not (2674,3872) but (2686, 3881) (0-indexed). When reading the image as read by imager (add 1, since R is 1-based) you get:

> im[2687,3882,1,]
[1] 0.1803922 0.4901961 0.3764706

which is the RGB value you expect.

Regarding the display, while I'm not completely sure, I'm half-guessing that display might be doing its own internal colour adjustment when displaying colours so that you get to see something, so I wouldn't give it too much attention.

Like @asgr says, I don't see an issue here.

[Gewerd-Strauss]

After a good amount of fiddling and more reading, I have successfully resolved the "issue", and will thus close this issue.
Thank you for the various explanations.