FR: RYB vectorscope

[phweyland]

Is your feature request related to a problem? Please describe.

The vectorscope is a powerful tool to analyze the colors present in an image.
It uses CIELUV and JzAzBz color spaces. These color spaces have been conceived to be perceptually uniform (delta E evenness).
For this reason there is no assumption nor relationship with color harmony.

Some color harmony rules has been described based on the RYB (or artist) color wheel (CIELUV and JzAzBz produce a color wheel close to the RGB one in term of polar coordinates).

RYB primary colors and the complementary ones

It should be noted that if RYB is said a substractive model, the YMC is the substractive equivalent to the additive RGB. So here we are not going to debate about additive / substractive property, but only about harmony ... which should be the same whatever the mean to obtain the final colors.

Based on the RYB color wheel we have some simple geometric rules which give (normally) pleasant color associations:

• monochromatic or closed colors (can be seen also on a RGB color wheel :))
• analogous complementary
• complementary (teal & orange)
• split complementary
• triadic
  ...

It's a bit strange (for me) to see that the same geometric rules are commonly applied on the RGB color wheel. Krita, for example, has an artist color picker, implementing these rules on RGB. That doesn't produce opposite results but the differences remained noticeable.
For my standpoint (I'm not artist ...) I prefer the RYB based harmony.

Describe the solution you'd like

If we believe that the rules which have been applied by the greatest artists during several centuries are still valid and can be applied on our images it could be worth to add a third vectorscope, RYB, beside the CIELUV and JzAzBz ones.

That should help to analyze and tweak an image based on these rules.
This has also been suggested in #9798 and #9209.

Alternatives

Some alternatives have been discussed on pixls.us in using-color-balance-rgb-for-creative-color-grading or create-custom-color-palette-to-show-in-the-vectorscope.
If the overall workflow is similar, I believe that a RYB vectorscope would simplify it.

I think that the RYB topic have poped up in the discussions based on this impressive video Secrets of color-grading in photography from Joanna Kustra.

Additional context

RYB is not a color space but there is some literature which can help to find a (kind of) RYB equivalent to the sRGB color space, where the harmony geometric rules are mode valid:
Paint Inspired Color Mixing and Compositing for Visualization (Gosset)
https://math.stackexchange.com/questions/305395/ryb-and-rgb-color-space-conversion/2776901#2776901, reversible (Jean-Olivier Irisson)
Computational RYB Color Model and its Applications
It is worth to cite the paletton site as a generator of complementary colors.

[Mark-64]

There are few FR on the vectorscope currently, directly or indirectly inspired by that video from Joanna Kustra.
On top of this one we have #9836, #9798 and maybe others.
Plus, there is already an ongoing heavy rework of vectorscope and color picker, see #9835 #9640.
I wonder if we can help here. I could try to implement some of the simpler FR (not this one), but I dont' want to interfere.

[dtorop]

If we believe that the rules which have been applied by the greatest artists during several centuries are still valid and can be applied on our images it could be worth to add a third vectorscope, RYB, beside the CIELUV and JzAzBz ones.

!!!

I appreciate the write-up. I'll be a bit swamped in the near term with various things (darktable and non-darktable) but if no one else claims this as a project, I'd be happy to make a try on it, probably later this year...

[phweyland]

I've started to look into it. Trying to understand how that works currently.
If I can I'll make a try. I'm on the background for the time being. Let's see if I succeed in doing something ... :)

[dtorop]

That's great! Indeed if you can get the background calculated in _lib_histogram_vectorscope_bkgd() you're 90% of the way there... You probably figured out that in _lib_histogram_draw_vectorscope() you can do something like:

modified   src/libs/histogram.c
@@ -936,10 +936,14 @@ static void _lib_histogram_draw_vectorscope(dt_lib_histogram_t *d, cairo_t *cr,
   if(!isnan(d->vectorscope_pt[0]))
     cairo_push_group(cr);
   cairo_set_source(cr, bkgd_pat);
+#if 0
   cairo_mask(cr, graph_pat);
   cairo_set_operator(cr, CAIRO_OPERATOR_HARD_LIGHT);
   cairo_set_source_rgba(cr, 1.0, 1.0, 1.0, 0.55);
   cairo_mask(cr, graph_pat);
+#else
+  cairo_paint(cr);
+#endif
 
   cairo_pattern_destroy(bkgd_pat);
   cairo_surface_destroy(bkgd_surface);

And then it'll draw the background to test that out. (Actually the above patch isn't working for me right now in CIELuv, not sure why...)

The code in _lib_histogram_process_vectorscope() could use another pass -- more optimizations and more brevity -- but that doesn't have to be your problem... It shouldn't be too difficult to add an extra vectorscope type to the UI...

Sometimes a test image as below can be nice, as it shows a full span of RGB values in the scope...

[phweyland]

And then it'll draw the background to test that out. (Actually the above patch isn't working for me right now in CIELuv, not sure why...)

I've discovered that. 👍 It works, at least I can see two different wheels for CIELuv and Jzazbz.

[phweyland]

Gosset RYB background with darktable and darktable-elegent-grey theme:

Is really convenient that the theme changes the graph colors ?

The same with gmic RYB:

I've not yet made work rgb2ryb() with gosset but the gmic algo seems to start to work, here the Joanna's color wheel (screen shot):

We can see that hues are not evenly spread. Here with gmic rg2ryb() and dt_RGB_to_HSV().

[dtorop]

It looks promising!

Is really convenient that the theme changes the graph colors ?

It's something to think about. The scope has a subtle background right now based on the graph_bg CSS. This seems to make sense for consistent theming (and gives a hint as to what's in/out of graphable range). It could be possible to render the graph to a temporary buffer (as already is the case for when there is a point sample overlay) and see if a compositing mode besides CAIRO_OPERATOR_ADD can produce a good result.

On the other hand, the key thing seems not so much that the background color looks the same for each theme, as that the distinction between hues is legible.

[phweyland]

With your color palette (@dtorop) and JzAZBz we get a nice spread of color among the gamut. Thanks for suggesting to use it :).

With gmic RYB = rg2ryb(RGB) and dt_RGB_to_HSV(RYB) we get this:

Applying a RGB transform to linear RGB just before we get this:

But apparently still with too much max saturation ... Hue seems to be coherent however.
I have the same saturation effect bypassing rg2ryb(), so this is not this piece which introduces it.

Is the HSV saturation reliable ? EDIT: if we divide the color chart in 4 vertical parts, the black and white ones give S = 0 and the colorful part gives S=1. The evenly spread S values come all from the light colors part.
Would a RGB -> XYZ -> HSV data flow be better ?
I've looked around to find XYZ -> HSV transform without success so far.

[dtorop]

Looks good! Are you sure it isn't simply done? The middle test image shows the full RGB colorspace, but it should like wrong in RYB, becuase RYB is "wrong" from the RGB POV. And you get to see something nice, how a quantized RGB space is getting skewed to talk about RYB.

It seems like the plan should be simply to take whatever RGB data you comes in with the histogram profile (may be linear, may be not), then convert to RYB, then use the HSV-ish code to bring it to polar coordinates, and that is it! Nice that the hue ring is round, this suggests that it is an "ideal" colorspace.

I don't think XYZ -> HSV makes sense, as HSV is another way of talking about an underlying RGB colorspace.

I could be missing details here, or the big picture...

[phweyland]

with the same color picker,
JzAzBz:

with RYB (HSV):

EDIT: I've checked the math, HSV really gives S=1.0 for that point.

[phweyland]

Using chroma instead of saturation from the HSL model I get this respectively for JzAzBz and RYB:

Full saturation has gone and the color picker position is coherent.
Note that red to yellow colors are expanded while green colors are compressed. in the RYB vectorscope.

[SoupyGit]

This is a really cool feature. If you want to take it one step further, it would be amazing to turn on/off guide lines of the different colour harmonies, such as Monochromatic, Analogous, Analogous Complementary, Complementary, Split Complementary, Triadic, Double Complementary, Dyad, etc...

I envisage these to be a thin grey lines going from central point to the outer ring, with user ability to rotate guide lines around the hue circle. Not sure how difficult that would be to implement, but it would be a cool feature.

[phweyland]

with Johanna's color wheel:

another color wheel picked up from web:

I think this comes from the gmic (Junichi SUGITA, Tokiichiro TAKAHASHI model), which covers the full rgb color pallet and is reversible.
The Gosset model should be closer to art pallets (but it excludes part of the rgb color range). Remains to make it work...

This is a really cool feature. If you want to take it one step further, it would be amazing to turn on/off guide lines of the different colour harmonies, such as Monochromatic, Analogous, Analogous Complementary, Complementary, Split Complementary, Triadic, Double Complementary, Dyad, etc...

Thanks. Agreed. Let's see if we can do something coherent. I'm not yet convinced about the usability. I've tested some to the Johanna's images and the vectorscope doesn't tell me exactly what I see on the images. But that's may be my own lack of competence. I'm learning...

[phweyland]

Gosset RYB (if no error in algos)

For reference, Gmic RYB:

But with Gosset the graph of RGB values is like that (some rgb values have not equivalent in ryb):

Other point for actual images.
Shadows, mid-tone and highlights chroma are displayed all mixed together.
Wouldn't the image graph reading be improved if one could display them separately ? This should be valid for all vectorscopes, shouldn't it ?

[dtorop]

To my eye, the Gosset looks much smoother/aestetic. It's at the cost of clipping the deep blue colors. Perhaps moving the blue point in Gosset slightly could help? The posterized transition at about 5 o'clock on the G'Mic from green -> cyan -> blue seems odd.

Shadows, mid-tone and highlights chroma are displayed all mixed together.

Very true, the nature of a chroma-only readout. A dream would be to show more of a 3D volume, with shadow/mid-tone/highlights along another axis...

[phweyland]

The posterized transition at about 5 o'clock on the G'Mic from green -> cyan -> blue seems odd.

That appears also on Gosset. With 96 hues it's still visible. Disappears with 192 hues. I don't understand this artifact. However with the graph masking and blending that should be hardly visible.

Perhaps moving the blue point in Gosset slightly could help?

Good idea. I'll make a try.

A dream would be to show more of a 3D volume, with shadow/mid-tone/highlights along another axis...

With the @aurelienpierre PR #9640 (preferences accessible locally to the module), we could add some parameters quickly accessible.
A first option would be to eliminate more or less shadows and/or highlights from the graph (easy).
A second one would be to show only a thin slice of a given value/lightness (graph and gamut)...

To simplify the color reading, Gosset suggests to distort the ryb colors (the graph) to converge towards primaries and first secondaries. I'm trying to code his kind of cubic interpolation...

[phweyland]

Gosset + Interpolation type A - t * (B - A) (similar to trilinear above).

Interpolation type A + t^2 * (3 − 2 * t) * (B − A), we get wider primaries + secondaries

But it also highlights the defects of the rgb2ryb model.

I agree that the Gosset background looks better. But to show the graph we need the rgb2ryb() function. And it does not look great, at least with the settings I'm using (and a bug is not excluded as well ...).
The inverse of ryb2rgb is discussed here.
Not sure we can improve it.

[dtorop]

Interesting! The "Gosset + Interpolation type A - t * (B - A)" looks good enough. Does it make those strange net-like overlaps as in your screenshot at the bottom, though?

[phweyland]

The "Gosset + Interpolation type A - t * (B - A)" looks good enough.

The background is built based the original Gosset RYB cube.

Does it make those strange net-like overlaps as in your screenshot at the bottom, though?

The graph is built a reversed cube, which has been proposed on the link above.
This reversed cube doesn't work as shown by the graphs below.
Only the Junichi SUGITA, Tokiichiro TAKAHASHI model (gmic) is fully reversible and covers the full range of RGB values. While the background is not as smart as the Gosset one, it ensures the graph is coherent.

With gmic, the RGB span is well represented (chroma coherent along radius (different lightness)):

With Gosset linear, see how the chroma varies with lightness:

Accentuated with Gosset cubic (the 6 vertexes should appear, only 3 does):

With a real image, referring to one of Johanna's video here:

JzAzBz

RYB gmic (not exactly the harmony presented by Johanna but close to).

RYB Gosset linear (new orange, more yellow, ...)

RYB Gosset cubic

[aurelienpierre]

I find concerning the fact that cyan is much more squeezed than other colors.

[phweyland]

cyan is much more squeezed than other colors

True. That's the cost of expending orange keeping red and blue at the same place. Green and cyan are squeezed.

Here is the pure RGB wheel:

The RYB gmic one:

The Gosset background does look better but the reverse function is problematic (probably because it changes not only the hue but also chroma and lightness).

[phweyland]

Idea: take the hues of Gosset vertexes, and find the corresponding reversible function(hue). It would less compress the cyan, but keep chroma and lightness... spline ?

[phweyland]

Gosset hues + spline: extension red -> yellow / compression yellow to red

or

depending on conditions around hue = 0 (I really don't master that point).

For the Johanna's image above: