Calling hsl function with a hex code is slightly off

[ndreas]

I stumbled upon this while tweaking https://github.com/npxbr/gruvbox.nvim which is based on lush.nvim. Using hsl('#aabbcc') results in a slightly different color in the highlight.

It was fairly easy to reproduce, I just ran :LushRunQuickstart, followed by :Lushify and changed the highlight of the search, followed by running :hi Search to see the difference:

As you can see, #cc241d becomes #cd231d in the highlight.

Is this by design? I'm no color expert but I seem to recall converting between RGB and HSL can be lossy. To me the difference is not so big that I notice it, but I kind of expect that colors set by hex RGB stays the same since :hi takes hex RGB codes.

[rktjmp]

Hmm... Yeah converting between RGB and HSL is sort of fuzzy because there's some amount of int -> float -> round -> int conversion conversion going on.

As you say, the difference is tiny, probably nearing imperceptibility for all but the lucky people on $40k displays?

#cc241d -> rgb(204, 36, 29)
#cd231d -> rgb(205, 35, 29)

I imagine if I were to find a way for it to be lossless here, it would just end up being lossy for another color (i.e clamp up vs clamp down).

I will leave this open until I can find time to check how much drift is in the conversion, maybe some colors have more. I imagine they are nearly all within a point of each other though.

[rktjmp]

Sorry meant to post back on this. I did investigate further. The nature of converting between RGB and HSL often leads to fractional HSL values (i.e. 32.42% saturation), which end up getting munged.

Even the HSLuv library, which at least gives the impression of being made by someone pretty "into colors" has the same effect.

I don't think it's actually a problem in use though, here's a random set of colors pushed through rgb -> hsl -> rgb:

#47FF35 -> #44FF33
#AA8B09 -> #AA8909
#B9B7FC -> #B8B6FC
#67CE69 -> #69CE6B
#172A54 -> #172A54
#FBBCC9 -> #FBBCC8
#CC7973 -> #CD7A74
#9BACA1 -> #9BABA1
#E1FDFB -> #E2FDFB
#F7980F -> #F8960D
#74EBCB -> #75EBCC
#0F8477 -> #0F8577
#06D970 -> #06DB70
#D97295 -> #DA7294
#C4FD0C -> #C5FD0D
#709C59 -> #709B59
#82D1A0 -> #80D09F
#35EBF6 -> #37ECF6
#6D11D4 -> #6C11D4
#384325 -> #374224
#CDFD51 -> #CCFD4E
#A1759D -> #A2769E
#A0D99F -> #A1D9A0
#229537 -> #229637
#CA79C2 -> #C978C1
#76D64F -> #74D54D
#62F184 -> #5FF181
#C529F9 -> #C529FA
#707C96 -> #6F7B95
#094830 -> #094930
#74C932 -> #73C832
#C76516 -> #C56516
#C05292 -> #C15393
#DE9823 -> #DE9921
#72513E -> #74523E
#661163 -> #641162
#F8C96F -> #F8C76D
#473CE6 -> #483DE6
#A8E187 -> #A9E189
#298C44 -> #288A42
#F4FB1A -> #F4FB18
#F7E1F9 -> #F7E1F9
#3BAF07 -> #3DB007
#132DBF -> #132DBE
#23E45B -> #25E45B
#7B1F9F -> #7A1F9E
#04CB15 -> #04CD15
#26E0B6 -> #24E0B4
#5DBB8D -> #5DBB8E
#85BF45 -> #84BF45
#B138E2 -> #B236E2
#9A6671 -> #996671
#D907AC -> #DA07AC
#C6A450 -> #C7A652
#FAF7BC -> #FAF7BD
#3DB0E8 -> #3BAEE8
#885B80 -> #8A5C81
#229240 -> #22913F
#BFC5E3 -> #BFC5E3
#04087F -> #040881
#5A63A8 -> #5B64A9
#6C37C3 -> #6D37C3
#BD41C8 -> #BD41C8
#FE5559 -> #FE5255
#377032 -> #387133
#CD8A4A -> #CE8A4B
#84A599 -> #84A498
#8BC6DE -> #8CC7DE
#A9025C -> #AC025C
#F5C602 -> #F2C202
#4FDCEA -> #4DDDEA
#E15E18 -> #E25F18
#9995A8 -> #9994A8
#13E8CB -> #13E7CB
#F20B23 -> #F40B23
#743F8C -> #743F8D
#5E84C6 -> #5D83C6
#6B0E32 -> #6C0E32
#4E24CF -> #5025D0
#C1486D -> #C1496D
#337C24 -> #327B24
#EF4413 -> #EF4415
#240F39 -> #240F38
#A44DF0 -> #A44DEF
#F6F2F4 -> #F7F3F5
#E96D6D -> #E96D6D
#8F9FE5 -> #8F9FE5
#BB3F80 -> #BB3E7F
#0A965B -> #09955B
#BE168F -> #C01690
#609452 -> #609451
#CB0335 -> #C90335
#E15FDF -> #E160DF
#7D2B67 -> #7D2B67
#D55C68 -> #D55D69
#FB5808 -> #FB5A09
#6372BA -> #6372BB
#26EC10 -> #26EA10
#5C9251 -> #5D9451
#6D4447 -> #6E4547

And a comparison of the first pair, I can't really spot the seam between the two, it will be monitor and eye dependent though. Maybe the bottom one is darker, maybe sort of? Maybe?:

Pretty interestingly, if you convert both RGB colors to HSL, you get the same values, so they are identical in the HSL colorspace:

They're actually equal in HSL, HSB and drift by 1 in LAB and CMYK.

I think you're more likely to notice differences when swapping terminals or monitors than the slight drift in RGB.

Going to close this as I think it's:

1. Maybe unsolvable? (If HSLuv has the same behaviour, I doubt I can come up with a better solution).
2. Not really impactful in use?

It is an annoying quirk though.

If you (or anyone) finds a colour that is obviously incorrect, definitely reopen.