[css-color-4] Missing info about Premultiplication and Undefined values

[facelessuser]

In this issue (#7253) a change was made :

If the alpha value is none, the premultiplied value is the un-premultiplied value. Otherwise,

I still feel this is a good call, but it is often referred that premultiplication happens before interpolation, which it does. Still, maybe it should be mentioned that at interpolation time, if given two colors (A and B), if one of the colors has an undefined alpha (we'll say A), and the other does not (B), then when A takes on the alpha value of B, it will need to premultiply on the fly before the actual interpolation takes place. This can be inspected and done ahead of time, but I think maybe it should be mentioned.

[svgeesus]

Re-reading the relevant sections Interpolating with Missing Components and Interpolating with Alpha I agree, it would be easy to miss that the handling of missing components happens before the alpha-premultiplication stage.

Which means that the only time alpha=none persists through to the premultiplication stage is when both colors have it; and in that case, the alpha premultiplication and un-premultiplication steps are both unity no-op.

Needs to be mentioned as you say, and ideally more examples added which show the interaction between the different stages.

[svgeesus]

I started working on examples and (once you consider nonsense examples, because none can occur anywhere) find I have an order of operations problem. Here is a dumb example:

--start: color(prophoto-rgb 0.3 none 0.7 /0.5);
--end: lab(100% 0 0 / none);
.foo {
  background: linear-gradient(in oklab to right, var(--start) var(--end));
}

Selected quotes from the spec:

For handling of missing component in color interpolation, see § 12.2 Interpolating with Missing Components

and

For all other purposes, a missing component behaves as a zero value, in the appropriate unit for that component: 0, 0%, or 0deg. This includes rendering the color directly, converting it to another color space, performing computations on the color component values, etc.

(both from 4.4 “Missing” Color Components and the none Keyword

Interpolation between values occurs by first converting them to a given color space which will be referred to as the 'interpolation space' below, and then linearly interpolating each component of the computed value of the color separately.

from 12. Color Interpolation

and lastly

If a color with a missing component is interpolated with another color which is not missing that component, the missing component is treated as having the other color’s component value.

from 12.2. Interpolating with Missing Components

So on the one hand, --end needs to get the alpha value from --start, 0.5, which is easy as they both have an alpha component in their source forms; but also, --start needs to get the green value from --end which is problematic because Lab doesn't have a green component.

And this filling in of missing components must necessarily happen before converting them to the interpolation color space, because color space conversion changes missing values to 0.

What I think is reasonable, then is that there is no green component to copy from; and thus --start is unchanged; and when converted to OKLab the missing component will become zero.

--start: color(prophoto-rgb 0.3 0 0.7 /0.5);
--end: lab(100% 0 0 / 0.5);

which is

oklab(38.63% 0.076 -0.27 / 0.5)
oklab(100% 0 0 / 0.5)

and interpolation proceeds as normal (both colors are premultiplied, components are interpolated, results are un-premultiplied).

[svgeesus]

One more gotcha: consider

--one: color(sRGB 0.7 0.4 0.2 / none);
--two: color(display-p3 0.8 0.7 0.4 / none);

both are missing alpha values:

If both colors are missing a given component, the interpolated color will also be missing that component.

Spec is missing some text to clarify that, when these two colors are converted to the interpolation color space, those two none values do not get converted to zero but must be preserved. Otherwise, the interpolated result will be fully transparent (alpha of zero) not alpha of none. And also, that zero alpha would give premultipled color components of zero, thus transparent black.

[svgeesus]

From my earlier comment I think that we need to expand the text slightly, from

If a color with a missing component is interpolated with another color which is not missing that component, the missing component is treated as having the other color’s component value.

to

If a color with a missing component is interpolated with another color which has that component, and the value is not missing, the missing component is treated as having the other color’s component value.

[svgeesus]

This will produce occasionally surprising results, where the two colors have matching components but the scales or offsets are different. For example copying OKLCH hue to an HSL color with missing hue gives you a result, but that color is not the same hue.

rebeccapurple converted to OKLCH oklch(44.03% 0.16 303.4) and converted to HSL is hsl(270 50% 40%)

Now interpolate oklch(44.03% 0.16 303.4) with hsl(none 50% 40%) and the second color becomes hsl(303.4 50% 40%)

rebeccapurple is rgb(40%, 20%, 60%)
hsl(303.4 50% 40%) is rgb(60% 20% 57.73%)

Or copying an OKLCH Lightness (range 0 to 1) to a CIE LCH Lightness (range 0 to 100) will work well if the percentage form was used, but oddly if the number form was used.

[facelessuser]

So, this is starting to get a bit confusing:

If a color with a missing component is interpolated with another color which has that component, and the value is not missing, the missing component is treated as having the other color’s component value.

So, are we comparing these colors before they are converted to the common color space? If I'm comparing an sRGB color with a Rec 2020 color that has different scaling, the sRGB color need to take on the red value from the Rec 2020 color? You seem to mention these surprising results with hues not being compatible and scales, I'm not sure I like this approach very much. Color spaces may share the same components, but they are not exactly the same in meaning. They may have different primaries for example.

I feel if people are trying to use none to their advantage, it would be best that they work with their colors in a consistent color space to take advantage of the none and avoid none to zero translation.

If the intent is to special case alpha specifically, I can more see that as a possibility. It is independent of color conversions and definitions of color spaces. I remember some suggesting the possibility of making alpha resolve to 1 instead of zero for instance. I could see an undefined alpha reasonably being treated specially during a conversion process and not altered, and then only at display time resolved to some sane, viewable value. I'm just not sure I personally see much of a benefit to complicating the conversion process for non-alpha components.

[svgeesus]

So, are we comparing these colors before they are converted to the common color space?

Right, otherwise all the none become 0

I feel if people are trying to use none to their advantage, it would be best that they work with their colors in a consistent color space to take advantage of the none and avoid none to zero translation.

Yes I think we are going to need some authoring guidelines here.

We added none everywhere (for consistency), when the actual motivation was powerless hue; now the spec of course has to say what happens in all cases many of which are far-fetched and are not goig to show up in real-world style sheets.

And yes people can use 'none` deliberately, for effect; but then they should be using a starting color space which is perceptually uniform and has semantically meaningful axes (so LCH or OKLCH, basically).

I'm just not sure I personally see much of a benefit to complicating the conversion process for non-alpha components.

I can certainly see that point of view.

[facelessuser]

Yeah, my two cents is that I'd prefer if color component translations stay with the current assumptions. It keeps things simple. Yeah, there can be weird cases, but if you are working in non-consistent color spaces, that's what you get 🙃. It seems more preferable than creating a more complicated conversion pipeline for interpolation, and there are still different, weird cases that are maybe less intuitive in nature.

If it is viewed that there is benefit in handling alpha differently, then I can maybe see that. I'm honestly kind of indifferent, but since alpha is agnostic to color spaces, you can kind of do whatever you want with it, if it makes more sense and provides a net positive benefit, I'm not sure I could argue.

[LeaVerou]

From a quick skim of the issue, here's what I see, correct me if I'm wrong:

Regardless of which components can have none values, when we're interpolating a color with none components into another color space, we have a problem. How will we handle e.g. hsl(none 0% 50%) when interpolated in srgb or literally any other color space that is not hsl? Are we going to now have entire expressions involving none values? Will we just coerce it to 0 and lose the noneness?

I wonder if it would be better to go back to just handling achromatic colors specially in the interpolation algorithm rather than have undefined components, even though undefined components is a more elegant solution... I mean, an achromatic color is sitll an achromatic color when converted to another color space…

[facelessuser]

I wonder if it would be better to go back to just handling achromatic colors specially in the interpolation algorithm rather than have undefined components, even though undefined components is a more elegant solution... I mean, an achromatic color is still an achromatic color when converted to another color space…

I'd personally hate the pendulum to swing too far in either direction. The heart of the issue had nothing to do with the current discussion. Really, I just thought it would be important to make clear that when two colors are being interpolated that premultiplication needs to come after an undefined resolution.

From the very beginning, it was understood that when a color with undefined components is converted that undefined values would need to be resolved, and it was decided that undefined channels would become zero. The spec already states this. This comes with certain implications that I thought were well understood.

Because none was introduced, some users could exploit this behavior and set channels to undefined values to try and take advantage of the logic, but as long as the spec makes clear what would happen in these cases, I think that is fine. I don't think the solution needs to be more magic or reverted.

I think the current behavior is fairly intuitive. Yes, I'm sure even if spelled out completely, some people may still be surprised at first, but that is true for a lot of things. If needed, I think it could be mentioned that if you are going to set none manually to channels in order to exploit its behavior during interpolation, undefined values will not persist through conversion unless the channel's undefined behavior is dictated by its achromatic nature. If you want the manual insertion of none to be meaningful, you need to work in the same color space, if not, they will be assigned a defined value of 0.

I think going back to only handling achromatic is the lesser of the two evils, but I'm not sure I prefer that over just adding a few more comments to make clear what should happen during premultiplication and will happen if you try to exploit none during interpolation.

[LeaVerou]

Maybe I'm missing something. How would you handle interpolating between e.g. hsl(none 0% 50%) and lch(50% 0% none) in e.g. OKLab?

The spec is very handwavy:

If a color with a missing component is interpolated with another color which is not missing that component, the missing component is treated as having the other color’s component value.

How do you even compute what "the other color's component value" is here, given they both have missing components so none can be converted to each other's color space?

[facelessuser]

Maybe I'm missing something. How would you handle interpolating between e.g. hsl(none 0% 50%) and lch(50% 0% none) in e.g. OKLab?

Yeah, I think the spec is unclear on this point. It does mention this:

If the colors to be interpolated are outside the gamut of the interpolation space, then once converted to that space, they will contain out of range values.

But it doesn't explicitly state that colors are first converted to the interpolation space before applying any undefined channel resolution.

It was my understanding that each color would be converted first to the interpolation color space (Oklab) and then have hue fixups applied, undefined channels resolved, and then premultiplication.

In the specific example posed in the question, none of the undefined channels persist after conversion to Oklab. Both colors convert to perfectly reasonable achromatic Oklab colors and the interpolation occurs without issues.

I think the spec could make this much clearer.

[LeaVerou]

My point is, you cannot convert to the interpolation space while having none components. Either you'd end up with expressions involving none (waaaaay too complex) or coerce none to 0 and lose the noneness.

[facelessuser]

While it may not be clear in the spec, I recall in prior discussions (maybe I am incorrect) that during conversions none values are replaced with zero.

[LeaVerou]

none is treated as 0 if the color needs to be displayed. It's unclear how conversion is supposed to happen, but I agree that would be the only reasonable thing to do.
Which means that if you interpolate hsl(none 0% 50%) to hsl(180 100% 50%) in anything other than hsl, you'd not get proper achromatic interpolation.
So not only is using none a very complex solution, it doesn't even address all the use cases.

[facelessuser]

none is treated as 0 if the color needs to be displayed. It's unclear how conversion is supposed to happen, but I agree that would be the only reasonable thing to do.

Ah, okay, then it must have been casually suggested at some time, but not formally made it into the spec, or maybe I inferred it as it made the most sense to me. That clears up some of the confusion though.

So not only is using none a very complex solution, it doesn't even address all the use cases.

I completely agree. It's kind of the reason I think trying to make them resolve perfectly in all situations (across color space conversion for instance) is not worth the effort, and there will always be some loss for any gain you try to compensate for.

I do think they have some use in interpolation, and I think that may be the only place. I think dropping them during conversion and clearly stating what happens to them is probably more than enough, but if a more drastic change was to be made, I think simpler is better.

Well, I've probably said enough 🙂 . I'll wait for whatever resolution is decided upon.

[svgeesus]

none is treated as 0 if the color needs to be displayed.

As I quoted earlier, yes.

It's unclear how conversion is supposed to happen, but I agree that would be the only reasonable thing to do.

Which part of Converting Colors is unclear?

[css-meeting-bot]

The CSS Working Group just discussed premultiplication and undefined values, and agreed to the following:

• RESOLVED: Specify how none is carried across color space conversion to a related component on the other side

The full IRC log of that discussion

<TabAtkins> Topic: premultiplication and undefined values
<TabAtkins> github: https://github.com//issues/7536
<TabAtkins> chris: we ahve some stuff abou thow to convert colors
<TabAtkins> chris: first, nones get replaced by zero, bc you need a number to convert
<TabAtkins> chris: we also have text to interpolate, which says if one has none it takes the value from the other
<lea> s/ahve/have/
<TabAtkins> chris: This avoids interpolating an achromatic to chromatic where it starts from an essentially random color
<TabAtkins> chris: These assume the space you're specified and interpolated in are the same
<TabAtkins> chris: If they're not, the conversion happens and you lose the info
<TabAtkins> chris: There's also the issue taht if you're using two different color spaces, what to do?
<TabAtkins> chris: If you have one in lch and one in hsl, do you copy the hue angle even if it's a different color?
<TabAtkins> chris: it starts not making sense unless the values you're comparing nones for are in the same color space
<TabAtkins> chris: then the question was, do we want this anyway?
<TabAtkins> chris: We started with saying tha tsometimes the channel was NaN
<TabAtkins> chris: and i was convinced to change it to none
<TabAtkins> chris: for consistency we added it everywhere, including in places no one asked for it
<TabAtkins> chris: that's extra testing, what's the point, etc
<lea> q+
<TabAtkins> chris: it does help i nsome cases - if you have two in the same polor color space, it helps
<Rossen_> ack lea
<TabAtkins> lea: for none to resolve, you clearly need to convert to the same color sapce
<TabAtkins> lea: no question
<TabAtkins> lea: but there's a question of how to convert if both colors can have none
<TabAtkins> lea: i don't think having none everywhere created new problems, they exist even if it's only present in hues
<TabAtkins> lea: we need to figure it out regardless of if it's allowed in rgb, etc
<Rossen_> q?
<TabAtkins> chris: i don't think that transporting nones from one color to another when they're in another color space is *useful*, it's just the only thing to do from the spec text
<TabAtkins> chris: but it doesn't make a lot of sense
<TabAtkins> florian: do you have an idea of the solution
<fantasai> TabAtkins: I disagree that it doesn't make sense when they're in different color spaces
<fantasai> TabAtkins: if you take achromatic hsl into chromatic lch
<fantasai> TabAtkins: it's ...
<lea> q+
<fantasai> TabAtkins: if you wanted to maintain intent of none behavior, I think answer is convert using the same rules, and remember the noneness and apply it on the other side of the conversion
<fantasai> miriam: that only seems possible with hue, though, right?
<fantasai> TabAtkins: powerlessness extends across conversions, right?
<fantasai> TabAtkins: if your hue is none and you convert to RGB ...
<fantasai> miriam: if your R, G, or B is none and you convert to hue, how do you do the conversion?
<fantasai> chris: to restate, you take which component names have none, then you do the conversion, and if that component name exists in the result ...
<fantasai> chris: if you convert hsl to lch, they both have hues
<fantasai> chris: but if you convert to profoto rgb, then no place to put it
<fantasai> TabAtkins: if you have zero chroma, should be same as ???
<fantasai> TabAtkins: so need to some manual mapping of what coverts to what as nones
<fantasai> TabAtkins: but e.g. polar to rectangular or vice versa wipes out the info
<fantasai> lea: even if going from hsl to lch, ...
<fantasai> lea: do we really want none^2/2?
<Rossen_> ack lea
<fantasai> TabAtkins: treat none as zero, convert it over, remember that the hue was zero, and then none it
<fantasai> lea: so would need to rmeember that hue in hsl and lch relate
<fantasai> TabAtkins: chroma and saturation should also map
<fantasai> TabAtkins: lightness should map
<fantasai> TabAtkins: if we want to maximize author friendliness of original intent, we need to set up a map of which channels can carry noneness into other profiles
<fantasai> lea: how would that work with custom profiles?
<florian> q?
<fantasai> TabAtkins: probably can't do it with custom profiles
<fantasai> lea: another suggestion is to ditch none altogether and handle achromatic colors specially, same as how we handle transparent colors specially
<fantasai> lea: it's not as elegan
<fantasai> TabAtkins: only objection there is one of the uses for none is to handle things without chroma that do have a definite hue
<fantasai> TabAtkins: e.g. things like white and black, they have undefined chroma as well
<fantasai> TabAtkins: so they don't go from zero chroma into white chroma red
<TabAtkins> s/white/bright/
<Rossen_> ack dbaron
<fantasai> dbaron: It would probably be useful to look the mapping from basically a table that shows what components in this color influence what components in ths other color
<fantasai> dbaron: for example, when convering hsl to lch, which components of hsl influence l in lch, etc.
<fantasai> dbaron: that might be useful
<fantasai> dbaron: it sounds like you want to transfer noneness in some cases beyond where it's strictly ok?
<fantasai> TabAtkins: less probably
<fantasai> dbaron: my intuitino is it would strictly be okay only where the inputs to the value were the ????
<fantasai> dbaron: I think
<fantasai> TabAtkins: might be, not 100% certain
<fantasai> dbaron: do more than that, but look at these tables and see what you want to do
<fantasai> dbaron: assuming you want to go down that path
<fantasai> chris: in genera, prefer ...
<fantasai> chris: but if I have a none alpha, and I convert to zero, and then I get transparent black, then ...
<fantasai> TabAtkins: we only premultiply during transitions right?
<fantasai> TabAtkins: idk if we've specifyed more clearly
<fantasai> chris: we have
<fantasai> TabAtkins: srgb is stored premultiplied?
<fantasai> chris: no, but when you interpolate you premultiply
<una> q+
<fantasai> miriam: but at that point you've already done the replacement
<fantasai> dbaron: is it possible you want the math for none to be different for different plcaes where you can put a none?
<lea> q+ to answer dbaron
<fantasai> chris: I think good thing about Tab's proposal is you don't have to handle none through the entire calculation chain
<fantasai> chris: just put it back at the end
<fantasai> dbaron: was suggesting for alpha cases, but maybe you want to treat it as 1 rather than zero
<fantasai> TabAtkins: yes, when you do conversoin you turn it into a number, and that's already in the spec
<fantasai> dbaron: ah, I thought someone said it's always zero
<fantasai> chris: it is
<fantasai> TabAtkins: even for alpha?
<fantasai> lea: maybe alpha should be 1
<una> q-
<fantasai> TabAtkins: alpha isn't involved in color space conversions
<Rossen_> ack lea
<Zakim> lea, you wanted to answer dbaron
<fantasai> lea: for HSL and LCH, for example, I think basically all components influence all components, it's jus thtat some components influence some other components more
<fantasai> lea: hue and lightness still influence every component, because not 1-1 mapping
<fantasai> dbaron: that was my intuition, especially once you're converting between d50 and d65 then you're definitely
<fantasai> TabAtkins: my intention was to put a table of what components map across, only a handful to worry about
<fantasai> TabAtkins: all the hue-ish things, all the red-green-blue-ish things'
<fantasai> Rossen_: that's the action out of this issue?
<fantasai> TabAtkins: proposed resolution is we specify how none is carried across color space converstion to a related component on the other side
<fantasai> Rossen_: objections?
<fantasai> RESOLVED: Specify how none is carried across color space conversion to a related component on the other side
<TabAtkins> ScribeNick: TabAtkins
<dbaron> (I do wonder how the stability of this relates to stability of other features in color 4.)

[facelessuser]

Hmm, I am worried a little bit about the resolution. This will definitely cause the intent of colors to change. There are certain cases that will turn out okay, but I think there could be plenty of cases where the intent of the color will be different.

We can see that for two RGB color spaces, preserving the NaN over each iteration changes the color:

The idea of handling hues (or I should be more specific, achromatic hues), I thought would get handled already which may be the main concern.

User agents may treat a component as powerless if the color is "sufficiently close" to the precise conditions specified. For example, a gray color converted into lch() may, due to numerical errors, have an extremely small chroma rather than precisely 0%; this can, at the user agent’s discretion, still treat the hue component as powerless. It is intentionally unspecified exactly what "sufficiently close" means for this purpose.

Note: As a reminder, if the interpolating colors were not already in the specified interpolation syntax, then converting them will turn any powerless components into missing components.

So, specifically tracking hues and translating them over to other cylindrical-based spaces seems unnecessary as that will happen during the conversion process for achromatic colors.

Alpha strikes me as the only other channel that stands out as a good candidate for maybe specially surviving conversion as it doesn't even contribute to the conversion. It could just be sidestepped in that process.

Anyways, maybe some of this info is helpful 🤷🏻.

[svgeesus]

TabAtkins: treat none as zero, convert it over, remember that the hue was zero, and then none it
lea: so would need to rmeember that hue in hsl and lch relate
TabAtkins: chroma and saturation should also map
TabAtkins: lightness should map
...
RESOLVED: Specify how none is carried across color space conversion to a related component on the other side

I just committed some spec prose and a table of analogous color components, plus an examle of carrying forward a missing value.

@tabatkins @LeaVerou @dbaron @facelessuser please take a look once the spec regenerates

[facelessuser]

I think the description makes sense, so from the angle of "does the spec make sense and is it clear?", I think the answer is yes.

I understand what its intention is, so I think it does that, but it comes at the cost of altering the color's original intent, which I'm still not sold on, but 🤷🏻. In achromatic cases, this should work out pretty much automatically, so I guess these downsides only come into play for non-achromatic colors that have users manually forcing a channel to none 🤷🏻.

On a side note, I want to mention this issue's main goal was just to make the spec clear on premultiplication, so I think that still needs to be updated. I don't want that to get forgotten as I feel this issue took a bit of a detour 🙂.

[svgeesus]

On a side note, I want to mention this issue's main goal was just to make the spec clear on premultiplication, so I think that still needs to be updated. I

Done!